Control unit operations in a real-time collaboration server

ABSTRACT

In a real-time collaboration server, a control unit manages a collaboration mode. The control unit operates a virtual client that maintains a virtual screen reflecting the status of the collaboration (e.g., the contents of a shared desktop or whiteboard). The virtual client renders collaboration data within the virtual screen. New clients are synchronized with an ongoing collaboration by packing and sending them a copy of the virtual screen. The control unit maintains a queue of collaboration data to be sent to participating clients. Each client may have a pointer identifying the queued data it is processing. The queue may be collapsed (e.g., when it reaches a maximum size) by sending a copy of the virtual screen to one or more clients that have not yet consumed old data in the queue; those clients are then updated to skip the queue entries embodied in the virtual screen.

This application is a continuation of, and hereby claims priority under35 U.S.C. §120 to a non-provisional application by the same inventor asthe instant application entitled, “CONTROL UNIT OPERATIONS IN AREAL-TIME COLLABORATION SERVER,” having Ser. No. 10/387,984, and filingdate 13 Mar. 2003 now U.S. Pat. No. 7,219,127.

BACKGROUND

This invention relates generally to the field of computer systems. Moreparticularly, a system and method are provided for operating a controlunit, in a real-time collaboration server, to share content from onecollaboration client with other clients.

Real-time collaboration systems are useful for sharing information amongmultiple collaborators or participants, without requiring them to bephysically colocated. However, existing systems have notabledeficiencies. For example, existing systems cannot readily accommodatelarge numbers of users (e.g., hundreds, thousands) while providingacceptable performance. Instead, as more and more users join acollaboration system, the response time (or other measure ofperformance) degrades rapidly until the system becomes unusable orbarely usable.

With some systems, the inclusion of just one slow user in acollaboration can bog down an entire system. This may be because thesystem enforces synchronous operations, in which each collaboration taskmust be completed before the next can be initiated. Or, a system maymanage collaboration communications very poorly. For example, a systemmay be able to store only a limited amount of data to be disseminated tocollaboration participants. When this maximum is reached, the system maybe unable to accept and disseminate new data until the slow userfinishes using the oldest data. Or, if the system can store an unlimited(or virtually unlimited) amount of data for dissemination, performancemay suffer as the amount of data that are stored increases.

Other collaboration systems require the processes or logic modules thatmanage a collaboration to also handle communication with individualusers. As the number of users grows, the burden of handling each user'scommunications as well as the collaboration logic becomes onerous.

Still other collaboration systems support only limited collaborationmodes (e.g., application sharing, document sharing), and/or attempt tomanage all modes with a single process. As a result, a mode that is notparticularly busy may be detrimentally affected by the activity ofanother mode.

An application sharing mode of collaboration may entail the replicationof content displayed on one user's computer upon other users' computers.One method of replicating content requires the interception of drawingand display commands executed on the first user's computer. Thosecommands are then replayed on the other users' computers. This method isgenerally limited to computer systems executing an operating system thatsupports such interception. In other environments, an interceptedcommand may not be supported or understood by the receiving user'scomputer.

Another method of replicating a computer screen involves capturing thefirst user's entire screen and sending it to the other users whenever itchanges. For example, the values of every pixel in the screen may betransmitted. This method is often very slow, and may require asubstantial amount of communication and processing for a small change(e.g., a cursor movement).

It may be difficult to add a new participant to an ongoing collaborationin an existing system. For example, the new participant may only beprovided the collaboration data that are generated after he or shejoins. These data may be meaningless without the context or status ofthe collaboration (e.g., the appearance of a shared document) at thetime the new participant joined. Or, if the system attempts to copy theentire status of the collaboration every time a new participant joins,system performance may suffer.

Further, in existing collaboration systems a new participant's clientoften must be rebooted in order to configure it for a collaboration.This is typically due to the manner in which updates to the client videodisplay are captured for use in the collaboration. Inefficient methodsof enabling video updates to be captured may also be unstable, dependingon how they perform the capture, or may be deactivated if a devicedriver is loaded or reloaded. Further, some methods only work forspecific operating systems or specific versions of an operating system.

SUMMARY

In one embodiment of the invention, a control unit within a real-timecollaboration server manages a collaboration mode. The control unitoperates a virtual client that renders collaborative data within avirtual screen. The virtual screen thus reflects the status of thecollaboration (e.g., the contents of a shared desktop or whiteboard).New clients are synchronized with an ongoing collaboration by packingand sending them a copy of the virtual screen. The virtual screen may beupdated synchronously (i.e., before other clients) or asynchronously(i.e., at substantially the same time as other clients).

The control unit maintains a queue of data to be sent to clients in thecollaboration. Each client may have an associated pointer or referenceto identify, within the queue, the data that it is currently processingor will process next. The control unit adds to the queue data receivedfrom a collaboration presenter, and retires queued data that have beenconsumed by all clients.

In another embodiment of the invention, a control unit's queue may becollapsed when it reaches a maximum size. In this embodiment, the queueis collapsed by sending a copy of the current virtual screen to one ormore clients that have not yet processed the oldest queued data. Thoseclients' pointers into the control unit queue are updated to skip thequeue entries embodied in the virtual screen sent to the clients. Oldqueue entries that are no longer needed for any clients can then bediscarded.

In one alternative embodiment, a control unit queue may be collapsedbecause of a slow client, even if the queue is not at its maximum size.Illustratively, a slow client is a client that is currently processingqueued data that most, or all, of the other clients have alreadyprocessed. In this embodiment, the control unit can, at any time,determine the amount of data or the number of queue entries that wouldhave to be transmitted to a client to update it to the current status ofthe collaboration. The queue may be collapsed if it would be moreefficient to send a client a virtual screen than to send the client allthe queued data that are reflected in the virtual screen but have notyet been processed by the client. In this embodiment, the currentvirtual screen may be assembled and sent, or a copy of an older virtualscreen may be sent.

DESCRIPTION OF THE FIGURES

FIG. 1 depicts a collaboration server for managing collaborations, inaccordance with an embodiment of the present invention.

FIG. 2 depicts an organizer for managing one or more collaborationsession, in accordance with an embodiment of the invention.

FIG. 3 is a block diagram of a control unit for controlling one mode ofa collaboration, in accordance with an embodiment of the invention.

FIG. 4 is a block diagram of a roster control unit configured tomaintain a roster of collaboration clients, in accordance with anembodiment of the invention.

FIG. 5 is a flowchart illustrating one method of adding a new client toa collaboration, in accordance with an embodiment of the invention.

FIG. 6 is a flowchart illustrating one method of receiving acollaboration communication at a control unit from a client, inaccordance with an embodiment of the invention.

FIG. 7 is a flowchart illustrating one method of sending a collaborationcommunication from a control unit to a client, in accordance with anembodiment of the invention.

FIG. 8 demonstrates the process of updating a shared desktop, accordingto one embodiment of the invention.

FIG. 9 depicts a control unit queue on which a form of queue collapsingmay be performed, according to one embodiment of the invention.

FIGS. 10A-B comprise a flowchart demonstrating a method of determiningwhether to collapse a control unit queue; according to one embodiment ofthe invention.

FIG. 11 is a block diagram of a client configured to participate in areal-time collaboration without having to first reboot, according to oneembodiment of the invention.

FIG. 12 is a flowchart demonstrating a method of configuring a client toparticipate in a real-time collaboration without first rebooting,according to one embodiment of the invention.

FIG. 13 is a block diagram of a desktop sharing engine according to oneembodiment of the invention.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the invention, and is provided in the context ofparticular applications of the invention and their requirements. Variousmodifications to the disclosed embodiments will be readily apparent tothose skilled in the art and the general principles defined herein maybe applied to other embodiments and applications without departing fromthe scope of the present invention. Thus, the present invention is notintended to be limited to the embodiments shown, but is to be accordedthe widest scope consistent with the principles and features disclosedherein.

The program environment in which a present embodiment of the inventionis executed illustratively incorporates a general-purpose computer or aspecial purpose device such as a hand-held computer. Details of suchdevices (e.g., processor, memory, data storage, display) may be omittedfor the sake of clarity.

It should also be understood that techniques of the present inventionmay be implemented using a variety of technologies. For example, themethods described herein may be implemented in software executing on acomputer system, or implemented in hardware utilizing either acombination of microprocessors or other specially designed applicationspecific integrated circuits, programmable logic devices, or variouscombinations thereof. In particular, the methods described herein may beimplemented by a series of computer-executable instructions residing ona suitable non-transitory computer-readable medium. Suitablecomputer-readable media may include volatile (e.g., RAM) and/ornon-volatile (e.g., ROM, disk) memory.

A system and method are provided for facilitating real-timecollaboration. In one embodiment of the invention, a collaborationserver is configured to serve large numbers of collaboration clients(e.g., thousands, tens of thousands), while supporting asynchronousoperations for any number of collaboration modes. In this embodiment,each mode supports a different type of collaboration (e.g., chat,polling, voice, annotation, sharing web content, collaborativewhiteboard, desktop sharing (i.e., sharing a client desktop)).

Control units control individual collaboration modes. For purposes ofcommunicating with control units, clients are represented by clientobjects. For each collaboration mode a client belongs to, its clientobject may include a data provider and/or a data consumer.

A data provider object feeds data from a client (e.g., collaborationdata to be distributed to other clients) to a control unit.Collaboration data are queued by a control unit and fed to clientsthrough their corresponding data consumer objects. Separatecommunication modules ensure reliable delivery of the data to eachclient.

In another embodiment of the invention, a method is provided for sharingcontent displayed on one client's desktop. In this embodiment, differenttypes of data (e.g., different graphical objects) or different types ofupdates to the desktop (e.g., different graphical operations) arecompressed or collapsed. Changes to the desktop content may be sent asincremental updates, thereby limiting the amount of information thatmust be disseminated.

In another embodiment of the invention, a control unit configured tocontrol a collaboration mode maintains a virtual screen tracking thestatus of the mode (e.g., the content within a shared desktop orwhiteboard, chat content). The virtual screen may be maintained by avirtual client, and sent to new clients that join the mode while it isin progress. The control unit may maintain a queue of data to bedisseminated to collaboration clients, and may collapse the queue whenappropriate. By collapsing the queue, its size can be controlled. Whenthe queue is collapsed, one or more relatively slow clients are sent acopy of the virtual screen and may thus avoid having to retrievemultiple sets of data from the queue.

In yet another embodiment of the invention, a real-time collaborationclient is configured to enable desktop sharing and/or other modes of areal-time collaboration without having to reboot. Illustratively, theclient is modified to intercept drawing commands and reduce changes tothe client's display into primitives that can be disseminated to othercollaboration clients.

A Real-Time Collaboration Server

FIG. 1 is a block diagram of a system for facilitating real-timecollaboration, according to one embodiment of the invention. In thisembodiment, collaboration attendees employ collaboration clients (e.g.,clients 102 a-102 m) to access collaboration server 110 via network 104or other direct or indirect communication links. A client may comprise adesktop computer, laptop computer, hand-held computer or other computingdevice. Network 104 may comprise the Internet.

Collaboration server 110 includes one or more web servers 112, anynumber of MX modules 114 (e.g., modules 114 a-114 n) and one or moreorganizers 116. Additional web servers, MX modules and/or organizers maybe spawned or initiated by a collaboration monitor (not shown in FIG. 1)or other module.

Illustratively, web server 112 receives connections from clients, andredirects those connections to MX modules. A web server may alsofacilitate the distribution of web page content to collaborationclients.

MX modules 114 route communications between clients, organizers and/orother nodes of a collaboration system, while organizers manage andmaintain collaboration sessions. The terms “collaboration” and“collaboration session” may be used interchangeably herein.

Each organizer 116 manages or controls one or more collaborationsessions hosted on collaboration server 110. Each collaboration mayinclude any number of modes. Each mode may be considered a separatetopic or support a separate purpose. Thus, a collaboration may include adesktop sharing mode (e.g., sharing one client's desktop with otherattendees), a whiteboard mode (e.g., sharing a virtual or electronicwhiteboard or drawing board), a chat mode, a music sharing mode, orother mode (e.g., polling, voice, annotation, web browsing).

Each MX module can handle connections from multiple clients on behalf ofany number of organizers. Therefore, clients connected to a particularMX module may participate in the same or different collaborations orcollaboration modes. An MX module maintains persistent connections withits connected clients, and may allocate one communication socket to eachclient and organizer that it supports.

In other embodiments of the invention, an MX module may serve as aconnection point for other objects or entities (e.g., a collaborationmonitor, a voice response unit, a collaboration recorder). The MXmodules supporting an organizer need not be colocated with the organizer(i.e., they may not be in the same enclosure).

In the embodiment of FIG. 1, for each attendee or client connected tocollaboration server 110, the MX module 114 handling the client'sconnection supports a virtual channel between the client and theorganizer 116 that manages the client's collaboration. The MX modulereceives and forwards communications between clients and the organizer.When a communication is issued from an organizer, each MX moduleconnected to one or more clients participating in the collaborationensures delivery of the communication to those clients. An MX module cansupport multiple virtual channels with a single physical connection orcommunication socket.

As described above, an organizer in collaboration server 110 handles oneor more collaboration sessions, which may include virtually any numberof attendees and may encompass one or more modes or topics. An organizerfacilitates real-time distribution of information within all of itsactive collaboration modes, to all specified attendees. An organizer maybe configured to record or archive some or all of the content for any orall collaboration modes. In one implementation, a recording module mayact as a collaboration client.

Organizers maintain states for their attendees and modes, and alsomanage their attendees' (or clients') permissions. An attendee'spermissions may be dynamically updated during a collaboration toindicate whether the attendee is permitted to make or receive changes toa shared desktop, a whiteboard or other object that is the subject of acollaboration mode. A permission may also indicate whether a particularclient is to receive all communications updating a collaboration, someparticular communications, whether an attendee may join some or allmodes of a collaboration, etc.

In one embodiment of the invention, separate virtual channels areestablished between an organizer and each client participating in theorganizer's collaboration. As described below, the organizer may includeobjects designed to read from and/or write to those channels in order toexchange data between the organizer and the clients. The MX modulesensure that each individual client receives the data, thereby relievingthe organizer of the burden of actively handling individual clientcommunications.

FIG. 2 is a block diagram of an organizer, according to one embodimentof the invention. Organizer 202 manages a collaboration having at leastone mode and any number of attendees.

Attendees are connected to organizer 202 via one or more MX modules 204.As described above, an MX module routes collaboration communicationsbetween nodes. Thus, an MX module provides means for multiplexing manyclient connections to one organizer, and for efficiently distributingcommunications to multiple clients. Communications may be passeddirectly between an MX module and a client node, or indirectly through aproxy (e.g., a firewall) or other entity. Similarly, a client mayconnect directly to an MX, may be redirected from a web server, as shownin FIG. 1, or may need to connect via a proxy.

In different embodiments of the invention, an MX module may multicast acollaboration communication to its connected clients, may send thecommunication to each client in a peer-to-peer manner, or maydisseminate the communication in some other manner.

Organizer 202 includes communication layer or API (ApplicationProgramming Interface) 210, which handles communications with clientsthrough MX modules 204 a-204 n. Thus, the communication layer mayinclude not only software executing on organizer 202, but alsocorresponding software executing on individual clients or other nodesthat communication with organizer 202. Control unit layer 220 includescontrol units for controlling individual modes of a collaboration.Filter layer 230 filters interaction between control units andcommunication layer 210.

Also within organizer 202, completion port 212 allows the organizer, oran entity operating in any layer of the organizer, to schedule and/orlearn of an event (e.g., completion of a task). When a task iscompleted, such as an outgoing communication is delivered, or anincoming communication is received, notification is made throughcompletion port 212. Every layer of organizer 202 may interact withcommunication port 212, work thread(s) 214 and/or timer(s) 216. Inaddition, other modules or entities in a collaboration server may use anorganizer's completion port.

Work threads 214, in the embodiment of FIG. 2, may comprise one or morethread pools for handling events received through completion port 212(e.g., scheduling or queuing communications within a collaboration).Different pools or sets of threads may be associated with differentcontrol units. Work threads may block as long as no events are waitingto be dequeued from the completion port. When an event or task isreceived through completion port 212, a work thread 214 is spawned orwoken to handle it. In this embodiment, rather than spawning a threadfor every client coupled to organizer 202, threads are shared and mayhandle a variety of events or tasks.

An event initiated by a module within the organizer (e.g., a controlunit) may have two parts: a request and a completion notification. Manyevents may involve the dissemination of collaboration information amongclients (e.g., a change to a shared document, desktop or othercollaboration object). Thus, the request portion of an event may includepassing the event to communication layer 210 for processing. When theevent is completed (e.g., all targeted clients have received thedisseminated information), the completion port notifies the initiator ofthe completion.

In an embodiment of the invention, events are handled asynchronously.That is, the receipt, processing, completion and notification ofdifferent events may be intermingled. The status of every event need notbe ascertained before another event can be initiated, and the status ofeach event active at a given time need not depend on the status of anyother event.

In this embodiment, every event passed to a completion port from withinan organizer has an associated programming object called a token. Whenthe event is completed, the system invokes a method provided by thetoken in order to identify or locate the initiator of, or an entityinterested in, the event (e.g., a control unit). The token may alsoembody the state of the event and uniquely identify the action(s) to betaken in response to completion of the event. This way, the initiatorneed not retain a plethora of state information regarding differentevents.

Timer(s) 216 may be configured to expire after a specified period oftime. In an embodiment of the invention, the firing of a timer may beused to trigger the queuing or dequeuing of an event by the completionport, or invocation of a related token.

For example, a timer may be started when a communication (e.g., a datapacket) is issued from a control unit for dissemination to one or moreattendees of the corresponding collaboration mode. If all consumers havenot reported receipt of the communication on behalf of their clients bya certain time, the timer may expire and some specified action may betaken (e.g., the communication may be dropped from any virtual channelson which it was not acknowledged).

Communication layer 210 may support multiple communication protocols.The communication layer (and/or some other entity in a collaborationserver) may also support or provide data encryption, tunneling,compression and other communication handling.

Organizer 202 also includes control unit layer 220, which comprises oneor more control units 222 (i.e., control units 222 a-222 i). Differentcontrol units control different collaboration modes. Thus, a web controlunit may control a collaboration mode for sharing web content. A desktopcontrol unit may control a collaboration mode for sharing an attendee'sdesktop. Other control units may be implemented for other modes, such aschat, polling, whiteboard, voice, etc. In alternative embodiments, morethan one control unit may manage a collaboration mode, or one controlunit may manage multiple collaboration modes. Control unit operationsare described in more detail below.

Between communication layer 210 and control unit layer 220 is filterlayer 230. Filter layer 230 is aware of which control units areimplemented and active in organizer 202, and directs incomingcommunications (e.g., collaboration data from clients) to theappropriate control unit. Illustratively, the filter layer may determinewhich control unit should receive an incoming communication by examiningthe communication, or by extrinsic evidence.

In one embodiment of the invention, filter layer 230 provides anabstract representation of each client participating in a collaborationhosted by organizer 202. Such abstractions are represented in FIG. 2 asclient objects 232. Because they represent interfaces with controlunits, they are depicted as straddling the boundary between control unitlayer 220 and filter layer 230, but may be implemented in either or bothlayers.

A client object may act as a data provider when submitting acommunication to the organizer as part of a collaboration, and/or act asa data consumer when receiving collaboration data on behalf of a client.Other entities within an organizer may also be represented by dataconsumers, such as a module configured to record a collaboration, avirtual client (described in a following section), and so on.

Other objects may be provided to read and write communications from/toclients' virtual channels. In FIG. 2, these objects are depicted asreader/writer objects 234, which serve as interfaces betweencommunication layer 210 and the filter layer. One reader/writer mayhandle one or more virtual channels.

In one embodiment, an organizer receives a collaboration communication(e.g., a data packet) through the communication layer when areader/writer object reads it from a client's virtual channel. A dataprovider corresponding to the client receives the communication andpasses it to the appropriate control unit. The filter layer may verifythe client's permission to send the communication to the control unitbefore the data provider takes action.

Conversely, when the control unit has a packet or set of data to send toa client (or multiple clients), it places it in a queue to make itavailable to clients. Each client's consumer will request the data whenit is ready (e.g., after the client receives the previous set of data).

Each consumer may have a pointer into the queue to track whichcommunication it will request next. These pointers may be updated by thecontrol unit each time a client's consumer requests a communication oracknowledges the previous communication.

The filter layer (e.g., a client's data consumer) will instruct areader/writer to write the communication on the appropriate virtualchannel. The MX modules to which clients corresponding to the consumersare connected will ensure delivery of the communication to each client.In one alternative embodiment of the invention, the functions of areader/writer object and a client object may be merged.

Outgoing data from a control unit may be cached (e.g., in acommunication layer, in a filter layer) until it has been dispatched toall clients via their corresponding MX modules. A communication layermay send collaboration data to each MX module only once. Thus, in thisembodiment, the amount of processing and communication activity withinan organizer or control unit is proportional to the amount ofcollaboration activity, rather than the number of attendees, and maytherefore scale quite well.

Organizer 202 may also include modules (not depicted in FIG. 2) forhandling other tasks, such as database access, collaboration monitoring,logging or archiving, recording a collaboration mode, authenticatingclients, authorizing a client's permission to send or receive acollaboration communication, etc.

In one embodiment of the invention, two or more levels of client orattendee authorization may be performed. For example, a first level ofauthorization may be carried out when a client connects to a real-timecollaboration server to join a collaboration session or tries to join aparticular collaboration mode. This level of authorization determineswhether the client is allowed to join, and may be performed by a filterlayer entity (e.g., an authorization module). If allowed, acorresponding client object is created for the client. As describedabove, the client object may contain data consumer and/or data providerobjects, and may contain separate consumer and/or provider objects foreach control unit that the corresponding client belongs to.

A second, more detailed, level of authorization may be performed withina mode of a collaboration. For example, different clients in onecollaboration mode may have different abilities (e.g., to contributedata, to participate in separate discussions in a chat mode). Thissecond level of authorization is therefore performed (e.g., by anindividual control unit or client data consumer) when a client attemptsto take some action within the mode.

One of the control units of each organizer (e.g., control unit 222 a inFIG. 2) is a roster control unit, and maintains a roster of attendeesparticipating in the organizer's collaboration. The roster control unitmaintains state for each attendee/client and may also handlecommunications with clients regarding permissions and states (e.g.,receives permission requests, sends permission grants).

In one implementation of the embodiment of the invention depicted inFIG. 2, one attendee at a time is the presenter for a collaborationmode. The presenter has control of the collaboration mode, and thus haspermission to change a collaboration object—for example, to write on awhiteboard, to alter a shared desktop or document, etc. The presentermay grant others appropriate permissions, and may relinquish his or herrole as presenter to another attendee.

In another embodiment, control of a collaboration object may be shared,so that multiple attendees may change or update it simultaneously ornearly simultaneously. However, changes may be assigned some serialorder by the control units responsible for the collaboration modes inwhich the changes are made. This may make it easier to determine whethera particular attendee has received all events that he or she should.

When an attendee initiates an “event” (e.g., a change to a collaborationobject), the attendee's client transmits the event to organizer 202(through an MX module 204) via a virtual channel. The organizer (e.g.,filter layer 230) receives the event and verifies that the sender haspermission to initiate the event. The event is then passed to thecontrol unit 222 corresponding to the collaboration mode in which theevent occurred. The control unit processes the event and forwards it tosome or all attendees of the collaboration mode, if necessary.

Different control units may receive events from, and distribute them to,different attendees or clients, depending on who is allowed to initiateevents (e.g., by altering a collaborative document). In oneimplementation of the embodiment of FIG. 2, a roster control unitreceives input from all attendees and can send communications to allattendees (e.g., to report the addition or departure of an attendee). Awhiteboard, web, desktop or similar control unit may receive changesfrom the presenter of each collaboration mode and send changes to otherattendees. A chat or polling control unit may also receive input fromall attendees.

A communication layer, such as communication layer 210 of FIG. 2,provides reliable, full duplex, stream-based connections with clients(e.g., via virtual channels), and performs flow control on thoseconnections. It serves as one endpoint to each virtual channel between aclient and an organizer.

When an organizer's communication layer is initialized, all nodes thatwill use the communication layer register themselves and receive uniqueaddresses or identities. Each client counts as a node, as well as eachorganizer and any other entities that will send collaboration data to,or receive collaboration data from, a control unit. When a new clientconnects to an organizer to join a collaboration session, it isregistered and assigned a unique identity. Multiple virtual channels maybe established between any pair of nodes.

A communication layer's connection to a client is reliable, in that thecommunication layer guarantees that data sent from one endpoint of avirtual channel will reach the other endpoint in the same order andwithout being corrupted.

Because an organizer's communication layer can call upon any number ofMX modules to handle the distribution of an event, a collaborationserver can easily scale to serve more and more attendees. Thecommunication layer will not be over-burdened, because it need notseparately transmit each event to every attendee (i.e., it may send anevent to an MX module only once).

In an embodiment of the invention, a collaboration communication (e.g.,an event from a client to an organizer, or vice versa) may be requiredto include two things—an identifier of the collaboration mode (orcontrol unit) to which it belongs to (e.g., to facilitate routing of thecommunication to the correct control unit) and an approval value.

In this embodiment, the approval value of a communication from a clientcomprises a value that had been assigned to the client before it sentthe communication. An approval value may facilitate authorization of aclient's ability to send or receive a communication. For example, eachapproval value may be unique, and represent a particular client'sauthorization to send data within a collaboration or collaboration mode.

A client's approval value(s) may vary over time (e.g., as it gains andloses authorization to send data). Thus, while the client is approved tosend events, it uses the last assigned approval value. When acollaboration server determines the client is no longer authorized tosend data, that approval value is cancelled, and data communicationsfrom the client will not be accepted until a new approval value isassigned and used.

In different embodiments of the invention, or in different collaborationmodes, a virtual channel between an organizer and a client may bepacket-based or stream-based. In a packet-based virtual channel, eachpacket contains the approval value and collaboration mode identifierdescribed above. In a stream-based virtual channel, the collaborationmode and approval value are identified when the stream is commenced.Thereafter, individual stream contents are not expected to re-identifythe collaboration mode or approval value. For example, in acollaboration involving the exchange of voice data (e.g., in GSMformat), it would be inefficient to include this information in everysmall data packet.

FIG. 3 depicts a control unit for controlling a collaboration mode,according to one embodiment of the invention. The control unit isresponsible for controlling the flow and distribution of data within thecollaboration mode.

In this embodiment, each control unit and collaboration mode operatesvirtually independently of any other control units and modes, and has afew primary tasks: receive and mix data (e.g., changes to shareddesktops, shared whiteboards, chat sessions) from data providers,distribute data to consumers, synchronize new attendees with an on-goingcollaboration mode, support clients/attendees having variouscommunication and processing capabilities, and so on.

In this embodiment, an organizer includes one or more data providerobjects representing clients that may submit data during a collaborationsession. The organizer also includes one or more data consumer objectsrepresenting clients that may receive data during the collaboration.Thus, a data provider may be seen as an abstract representation of anynumber of clients/attendees that may submit communications to anorganizer or control unit. A data consumer may be seen as an abstractrepresentation of one or more clients/attendees that may receivecommunications from an organizer or control unit.

In FIG. 3, control unit 302 controls a collaboration mode that requiresinput or changes from one or more data providers 312, and yields ordisseminates output to one or more data consumers 314. The collaborationmode may be stateful, and each input may therefore be an incrementalupdate to the previous state (e.g., until the mode is reset).

In control unit 302 of FIG. 3, monitor 304 maintains the state of thecontrol unit's collaboration mode, and may store collaboration inputand/or other events for dissemination. Thus, as described above, monitor304 may implement a queue for collaboration data to be disseminated toclients.

And, as described in a subsequent section, a collaboration server (e.g.,a control unit such as control unit 302) may maintain (e.g., in monitor304) a current view or snapshot of a collaboration mode. This snapshotmay be considered a virtual collaboration status or a virtual screenintended to mirror the information shared by a collaboration presenter.When a new attendee joins the mode while it is in progress, the view orsnapshot can be used to facilitate synchronization of the new attendeewith the mode.

In an embodiment of the invention, the speed or rate at whichcollaboration input (e.g., data) is provided to a control unit may notmatch the speed or rate at which the control unit can disseminate eventsto attendees. For example, client network connections may be capable ofa wide range of maximum data rates, some clients may have less powerfulprocessors than others, and so on.

If a data provider attempts to provide data to a control unit at a ratefaster than all consumers can receive or retrieve traffic, some data mayneed to be buffered on the collaboration server. To prevent bufferoverflow in the server, the production or acceptance of collaborationevents may be suspended while previous events are disseminated. Or, thedata and/or other events of a control unit needed by a particular (e.g.,slower) client may be collapsed or condensed to allow the client to bebrought up to date more efficiently, as described in a later section.

Thus, in one embodiment of the invention, an organizer's control unitlayer or individual control units may limit the rate at which incomingchanges or collaboration input are accepted, in order to align the rateof data receipt with the rate of data dissemination.

The role of a control unit, such as control unit 302 of FIG. 3, can nowbe described for different collaboration modes, as they may beimplemented in different embodiments of the invention. One type ofcollaboration mode involves desktop sharing. In this mode, multipleattendees cooperate and share the state of the collaboration among theirdesktops. One attendee acts as presenter at a time—the attendee whoseclient desktop is being shared—and any number of attendees mayparticipate. Every collaboration update comprises an update to thepresenter desktop, or information enabling another client to replicatethe update.

The state maintained by the corresponding control unit (e.g., in monitor308 in FIG. 3) may include the current state of the desktop as a“virtual screen.” When a new attendee joins the collaboration, thecontrol unit sends the new attendee's client the virtual screen at aselected time, and a consumer may be added for the new client. Further,if a slow client was unable to accept collaboration events and data asfast as other clients; that client could receive updates at a slowerpace. Such updates may be snapshots of, or updates based on, the controlunit's virtual screen, or subsets thereof. Use of a virtual screen isdescribed in more detail below.

Another type of collaboration employs a shared whiteboard. This mode mayinclude multiple presenters and multiple other attendees. Every eventmay include a new drawing or drawing component (e.g., a vector, apolygon), a change to an existing sketch, a command to undo a previouschange or to clear the board, etc. The control unit for thecollaboration mode may maintain the current state of the whiteboard. Newattendees may be synchronized as described above for a desktop sharingmode.

Another type of collaboration mode may encompass audio or other mediasharing (e.g., voice, music, video). The presenter in this mode may be aserver or other apparatus that retrieves or provides the audio input.Each portion of audio is disseminated to all participating attendees.The control unit for audio sharing may not record a state, as a newattendee may be satisfied with just receiving all audio generated afterhe or she joins. In this example, if an attendee's client could nothandle the native data rate of the audio input, the control unit maydiscard audio data as necessary to accommodate the client's data rate.

A control unit may apply business logic and/or security logic todetermine whether and when a particular attendee should receive acollaboration event (e.g., a change or update to a collaboration objectsuch as a desktop or whiteboard). For example, attendees may requirespecific permissions in order to receive an event. A presenter of anevent may identify an attendee that should or should not receive anevent or set of events, and the control unit managing the collaborationmode will track that attendee's permission(s).

As described above, in one embodiment of the invention, everycollaboration includes a “roster” collaboration mode, under the controlof a roster control unit. The roster control unit deals with informationthat may affect an entire collaboration, not just one particular mode.For example, the roster control unit may manage the clients'/attendees'statuses as presenters and participants. This may entail managing theirpermissions to send and/or receive collaboration data and other events.

FIG. 4 depicts a roster control unit according to one embodiment of theinvention. In this embodiment, roster control unit 402 receives data(e.g., permission requests, state changes) from data providers 412, onbehalf of any or all of the clients participating in a givencollaboration. The roster control unit sends data (e.g., permissiongrants and denials, collaboration data, approval values) to dataconsumers 414 on behalf of their corresponding clients of the currentcollaboration.

Roster control unit 402 includes collaboration roster 420, which may bemanaged by a monitor module, as described above in conjunction with FIG.3.

Collaboration roster 420, when implemented in table format, includes arecord for every client participating in the collaboration. Each clientis identified by client ID 422, which comprises a unique identifier.

Each client's record also includes a set of client properties 424 and aset of permissions 426. Illustrative properties include a username,client location (e.g., IP address), indications of which collaborationmodes the client's attendee is participating in, etc. Illustrativepermissions include permissions to send data (e.g., changes to a sharedcollaboration object such as a whiteboard drawing or a desktop, acomment in a chat), receive data (e.g., for a specified collaborationmode), grant permissions to other attendees, etc.

In an embodiment of the invention, a permission comprises a permissionidentifier and a corresponding permission value. A permission identifieruniquely identifies a permission among all permissions applicable forall modes of a collaboration session. A permission value is a value(e.g., an integer, an alphanumeric value) indicating the status of thatpermission for the corresponding attendee or client. Illustratively,permission values indicate a status of Denied, Requested or Granted. IfDenied, the attendee has been denied the permission; if Requested, theattendee was denied the permission, but has now requested it be granted.

If Granted, the attendee has been granted the permission. In thisembodiment, when a permission is granted, the permission value comprisesan approval value, which a client must submit to the collaborationserver whenever it exercises the permission. When a once-Grantedpermission is subsequently denied, the assigned approval value becomesinvalid.

Roster control unit 402 of FIG. 4 also includes collaboration-wideproperties 430, which include properties of the overall collaboration.These properties may identify what collaboration modes are established,how many attendees are participating, which attendee is the presenterfor each mode, a collaboration title or name, some configurationinformation, etc. In one embodiment, collaboration roster 420 includescollaboration-wide properties 430.

When a collaboration communication (e.g., an event, a change to acollaboration object) is received at a collaboration server, it isforwarded to the control unit that controls the operative collaborationmode. However, before the control unit processes the communication, thecommunication is examined to ensure that the sending client hadpermission to send it. Thus, the filter layer, or some other object ormodule, may examine an approval value included in the communication. Inone embodiment, a client object (e.g., a data provider) representing theclient from which the communication was received verifies the client'spermission.

Illustratively, a collaboration roster or a separate verification modulemay be consulted to verify that the received approval value matches theclient's current approval value. Illustratively, components of anorganizer can access a collaboration roster without using the controlunit interface that clients must use. Thus, when a client object needsto verify a client permission, it can query the collaboration rosterdirectly.

Some communications may be accepted without verifying the sender'spermission. For example, in a chat mode, all registered attendees may beassumed to have permission to provide input.

When the collaboration server prepares a communication to one or morecollaboration clients, some or all clients' permissions to receive thecommunication may be examined by the filter layer (e.g., the clients'corresponding client objects) or some other object or module. However,in the interest of efficiency, some communications may be distributedwithout verifying permissions. For example, it may be assumed that allattendees registered to receive an audio stream, or other media stream,have permission to receive a normal portion of the stream.

From a control unit's point of view, a client's virtual channel can haveone of two statuses in each direction (i.e., send and receive): blockedor ready. At any given time, the channel may be blocked in one or bothdirections because a communication (e.g., a packet) is being sent orreceived over the channel. If not currently busy, then the channel isready to carry a communication to the client or receive one from theclient.

In an embodiment of the invention, an organizer's completion portcomprises what may be considered an I/O (input/output) event queue. Asclients acknowledge receipt of communications from a control unit ororganizer, and send communications to a control unit, the event queue ispopulated.

A control unit may maintain a queue, list or other structure fortracking communications dispatched as part of its collaboration mode.The control unit may maintain pointers, for each of its attendees'clients, to identify which communication(s) each client has or has notyet been sent or acknowledged.

When a communication is dispatched to a client, the operativereader/writer object may be marked as busy, and may not be returned tothe ready state until an acknowledgement of the communication isreceived and processed. Illustratively, notification of the client'sacknowledgement is received at the organizer's communication layer,through the MX module to which the client is connected. A work threadreads the notification and updates the sending control unit's status forthe client.

Work threads may operate in cycles, and may sleep when there are nocompletion events to service. When the I/O event queue has an event toservice, a work thread reads it and performs the necessary action. Asdescribed above, the event may include a token that specifies the actionto be taken.

When a control unit prepares or receives a new communication to be sentto its attendees, it may immediately make the communication available toclients. It then releases the communication (e.g., passes a reference tothe communication) to each client object when it requests the next datain the collaboration after having consumed the immediately precedingdata.

When a control unit releases a communication for transmission tomultiple clients close in time, the communication layer serving thecontrol unit, and/or the MX modules to which the clients are connected,will try to optimize the transmission. Thus, the communication may besent via multicast if possible.

Each MX module maintains a queue, buffer or other structure of thecommunications it has been instructed to transmit, and waits for thecorresponding clients' channels to become ready to receive. When aclient's virtual channel becomes ready to receive a communication, theMX module will transmit on that channel if it has anything for thatclient. A communication will be flushed from the MX module's buffer whenall clients have received it.

FIG. 5 demonstrates a method of connecting a new attendee to acollaboration session, according to one embodiment of the invention.

In state 502, a new attendee connects to a collaboration web server,through his or her client computing device (e.g., with a suitablebrowser). A listener in the web server accepts and forwards theconnection to an available MX module. A communication layer of theorganizer managing the target collaboration establishes or accepts avirtual channel between the organizer and the client.

In state 504, the client is authenticated, through PKE (Public KeyEncryption) or other means. The organizer may include an authenticationmodule configured to authenticate new clients.

In state 506 after the new client is authenticated, a unique client IDis assigned to the client.

In state 508, a client object is created in the organizer to representthe new client. As described above, a client object may reside in, orinterface with, a filter (e.g., filter layer 230 in FIG. 2). The clientobject may include a provider and/or a consumer, to represent the clientwhen it sends or receives communications to/from the organizer.

In state 510, the new client is added to a roster maintained by a rostercontrol unit. Illustratively, the roster control unit creates a clientrecord comprising the client's ID, any relevant properties associatedwith the client, any permissions the client is granted, etc.

In state 512, the new client is added to any collaboration modes theattendee requests or is assigned to (e.g., desktop sharing, whiteboard,chat, application sharing). The client's entry in the roster controlunit is updated as the client's properties and permissions change, andis removed and/or archived when the client disconnects.

FIG. 6 demonstrates a method of accepting data from a clientparticipating in an existing collaboration, according to one embodimentof the invention. The data may comprise a change to a collaborationobject (e.g., an update to a shared desktop, a modification to a sharedwhiteboard), a permission request, other state information, etc.

In state 602, a collaboration communication is received at acollaboration organizer (e.g., at the communication layer) from anexisting client, over the client's virtual channel. In one embodiment ofthe invention, the communication is received by the client object thatwas created when the client joined the collaboration. In thisembodiment, the client object acts as a data provider (to provide datato a control unit) when accepting data from its client. In anotherembodiment, reader objects are assigned to virtual channels to readcommunications received on the channels.

More specifically, a reader/writer object (e.g., within a filter orcommunication layer) may read the communication from the client'svirtual channel. The reader/writer then passes the communication or itscontents to the client's data provider.

In state 604, the communication is read from the channel, and theclient's ID and approval value may be retrieved from the communication.The contents of the communication may be queued, cached or buffered.

In state 606, the sending client's authorization to send thecommunication is verified. The verification may be made based on theapproval value retrieved from the communication. This action may requirereference to a roster control unit, a verification module or some otherentity within the responsible organizer.

If the client did not have authorization to send the communication, instate 608 the communication is discarded and the illustrated methodends. The originating client may be notified that it did not havepermission to send the communication.

However, if the client had permission to send the communication, instate 610 the responsible control unit is identified. Illustratively,the corresponding client object or a filter may retrieve an identifierof the control unit (or the corresponding collaboration mode) from thecommunication. For example, the communication may include a type fieldthat identifies the collaboration mode and/or control unit.

In state 612, the communication is forwarded to the appropriate controlunit. In the illustrated method, the contents of the communication mayfirst be received by the data provider (e.g., client object)representing the sending client. The provider notifies the control unitthat it has data for the collaboration mode. When the control unit isready, it requests or reads the data from the provider. When donereceiving the data, the control unit may instruct the provider toretrieve the next communication or set of data from the virtual channel.When a virtual channel is quiescent, the corresponding client object isidle.

In response to the receipt of data from a provider, the control unit maydisseminate the data to other attendees, may use it to update the stateof a collaboration mode or the sending client, or take other action onthe basis of the communication. For example, the control unit may queuethe data for transmission to clients participating in the collaborationmode. When a client can receive the data, its client object (acting as adata consumer) may then request it. The illustrated method then ends.

FIG. 7 demonstrates a method of transmitting data to a clientparticipating in an existing collaboration, according to one embodimentof the invention. The data may comprise a change to a collaborationobject (e.g., an update to a shared desktop, a modification to a sharedwhiteboard), a permission grant or denial, other state information, etc.

In state 702, a control unit for a collaboration mode assembles orprepares data to be sent to one or more clients. The data may have justbeen received from a presenter of the collaboration mode.

In state 704, a client object acting as a data consumer requests the newdata be sent to its client. Illustratively, a number of client objectsmay request the data soon after it is queued. As described in afollowing section, the dissemination of data to clients may becontrolled in order to manage the size of a control unit's output queue,handle slow clients, help synchronize clients joining a collaborationmode that is in progress, or for other purposes.

In response to a first request, in state 706 the control unit forwardsthe data to a filter or communication layer, where it may be cacheduntil no longer needed (e.g., until it has been sent to all clients).

In state 708, a determination is made as to whether a consumer's clientis permitted to receive or read the communication. This may requirereference to a roster maintained by a roster control unit, averification module within the organizer, or some other entity.

If the client is not authorized to receive the communication, in state710 the client is dropped from the scheduled recipients of thecommunication, but it may still be sent to other, authorized, clients. Aclient may also be dropped from the list of recipients if the client hasdisconnected from the collaboration server or closed the virtual channelfor the control unit's collaboration mode.

In state 712, the communication is forwarded to one or more authorizedclients. Each data consumer may update a pointer or other reference tothe control unit's queue, to reflect consumption of the latest entry.Similar to the reader objects described above, writer objects may beassigned to write communications onto virtual channels. Thus, state 712may involve identifying the writer for a client's virtual channel andinstructing that writer to send the communication. One object maycomprise both the reader and writer for a virtual channel and, in onealternative embodiment of the invention, may also comprise the dataprovider and data consumer corresponding to the client connected to thevirtual channel.

Desktop Sharing

In one embodiment of the invention, a collaboration mode involvessharing all or a portion of a presenter's desktop (i.e., contents of thedisplay of a presenter's client computing device) with one or morecollaboration attendees. In other embodiments, other types of data maybe shared as described in this section.

In a desktop sharing mode of collaboration, an initial version or copyof the presenter's desktop is sent to the attendees' clients, followedby incremental updates. Thus, when the desktop sharing mode isinitiated, the presenter's desktop (or a specified portion of it) as itappears at that time is reproduced on the attendees' clients.

Thereafter, changes or updates to the presenter's desktop are identifiedand sent to the clients in cycles. For example, a cycle may comprise asearch for incremental changes, generation of corresponding primitives(including any necessary management of cached objects) and transmissionof the primitives. Cycles may be initiated on a time basis (e.g., every200 ms), in response to a threshold number of events that affect thedesktop, after a number of presenter inputs, or on some other basis.

In an embodiment of the invention, updates and changes to thepresenter's desktop are transformed into object primitives representinggraphical objects or tools, and action primitives representingoperations upon or using those objects or tools. The primitives aredisseminated to attendees' clients so that they may reproduce theupdates. Illustratively, the primitives are sent from the presenter'sclient to a real-time collaboration server, and from the server aredisseminated to the attendees.

The primitives are designed to keep attendees' desktops very similar to(or exactly matching) the presenter's, while passing relatively smallamounts of information. The information that is passed may be compressedand/or encrypted. Illustrative object primitives (e.g., drawing objects)are listed in Table 1, and illustrative action primitives are listed inTable 2. Other object and action primitives may be employed in otherembodiments of the invention.

TABLE 1 Object Primitive Description Character A connected one-colorgraphic pattern or shape Cursor Pointing cursor (e.g., for a mouse)Image Image data (may be encoded in a variety of formats) Palette A setof predefined colors Region An area of the desktop

TABLE 2 Action Primitive Description Character update Draw a specifiedgroup of character objects at a specified location in the desktop Imageupdate Draw an image at specified coordinates in the desktop Source blitCopy the identified portion of the desktop to a specified area Tile Filla specified area with the specified tile pattern

Some or all object primitives may be cached for reuse, in a cache thatis replicated from the presenter's client onto the other attendees'clients. When an object is cached, a cache ID is assigned at thepresenter's client and accompanies the object to the other, clients.When a later action or operation using the same object primitive isdispatched to the other clients, it may include just the cache IDinstead of the whole object. In one implementation, all objects may becached. In another implementation images and/or other objects may not becached.

Illustratively, the presenter's desktop, or the portion of it beingshared with the attendees, may be logically divided into “clusters” thatare scanned for object primitives and/or other patterns or attributes.For example, the shared desktop may be divided into fixed-size squaresor rectangles (e.g., 16 pixels by 16 pixels). Not every cluster may bethe same size or shape.

Each cluster may be scanned with a specified regularity (e.g., every 200ms), whenever a display driver or operating system reports that thedisplay has changed, etc. In one embodiment, a checksum may be computedon some or all clusters every cycle, and only clusters whose checksumsdiffer between successive cycles may be scanned for changes.

One type of object primitive in Table 1 is a “region.” A region mayencompass the entire shared desktop or any portion thereof, and need notbe contiguous. For example, all portions of the presenter's desktop thatcomprise only a background (e.g., the desktop “wallpaper”) may compriseone region. Thus, a region is not necessarily a drawing object, but moreof an area in which to apply one or more actions or operations (e.g.,place a cursor, insert a graphical object). A region may be defined, ina region object primitive, by any suitable means (e.g., x and ycoordinates, vectors defining the boundaries).

In one embodiment of the invention, regions are formed from clustersafter they are analyzed. More specifically, when a cluster is analyzed,various attributes are assigned to or associated with it. Suchattributes may include the number of colors in the cluster, the numberof pixels of each of those colors, a ratio of the number of pixels ofone color to the number of pixels of another color, etc. Depending onthe analysis of a cluster, it may be determined that the clustercontains a particular type of object (e.g., character, cursor, image).

Clusters having the same or similar attributes (e.g., same colors,almost the same ratios of pixels of different colors) may then be mergedto form “regions.” In another embodiment of the invention, regions arepre-defined, and may or may not be used in place of clusters.

A “palette,” in this embodiment of the invention, also is not a drawingobject. A palette comprises a set of colors that may be used to describea character, image, or cursor to facilitate its reproduction onattendees' clients. A palette may identify any number of colors.

Illustratively, the system may attempt to match each set of colorsencountered in a cluster with a palette. If a palette with the samecolors does not already exist, it may be created. Or, if the colors area subset of an existing palette, that palette may be associated with thecluster.

By predefining one or more palettes on the attendees' clients, colors ofobjects in the presenter's desktop can be described with less data.Instead of having to send the RGB value of each pixel of an object, forexample, each pixel can be described with an index into the appropriatepalette. And, a palette may be defined to include just those colorsactually being used in the desktop or a region of the desktop, ratherthan the full color depth.

A “character” object primitive is not necessarily a member of the set ofalphanumeric characters. In this embodiment, a character may be anyconnected pattern of pixels having a single color. Thus, within ascanned region, any uni-color pattern of pixels different from thebackground color may be defined as a character. Such patterns mayinclude an alphanumeric character, a portion of an alphanumericcharacter (e.g., if the boundary between two adjacent clustersintersects the character), a symbol, a one-color figure (e.g., an icon),etc.

A “cursor” object primitive may be the cursor of a mouse (or otherpointing device), of virtually any shape. By defining a presenter'scursor as an object primitive, the cursor can be correctly and easilyplaced by caching the cursor object and simply specifying the positionat which to draw the object. To fully define a cursor object, its imagedata (e.g., shape, color), its size and its hotspot coordinate (i.e.,location) are noted. The hotspot coordinate may be used to track thecursor as it moves.

An “image” object primitive may comprise any graphical content thatisn't a “character,” “cursor” or other already defined object. An imageprimitive may be encoded in virtually any manner (e.g., according to anycoding scheme) for dissemination to attendees.

In one embodiment of the invention, a separate cache is maintained onthe presenter's client, and reproduced on each attendee's client, foreach type of object primitive (e.g., character, cursor, palette).Whenever an object primitive is first sent from the presenter's client,it is accompanied by its ID in the corresponding presenter cache. Anattendee's client adds the new primitive with the specified ID. In thisembodiment, the client may only discard a cached item when specificallyinstructed to do so (e.g., by the presenter client, by a collaborationserver).

Besides one or more caches of object primitives created by a presenter'sclient, a surface cache used by the presenter's operating system mayalso be replicated at the other clients. Thus, when the presenter'soperating system employs a graphical object (e.g., an icon) from itssurface cache, the object and surface cache ID may be passed to theother clients so that they can reproduce the surface cache being usedfor the shared desktop. Parts of a surface may be sent (e.g., ondemand); the entire surface need not be sent at once.

Regarding the action primitives of Table 2, according to one embodimentof the invention, “character update” action primitives may be used todraw character object primitives on attendees' clients. A characterupdate identifies a set of character object primitives (e.g., by theircache IDs) and their positions relative to, or within, a particularregion. All characters in one set may be of the same color. When thecharacter update primitive is executed, the specified set of charactersis drawn at the specified location.

Illustratively, the position of each character in the set may be definedby horizontal and vertical offsets (e.g., from the previous character inthe set). For example, the horizontal offset may be measured from oneboundary (e.g., the right-most point) of the previous character, and thevertical offset may be measured from another boundary (e.g., thebottom-most point). The first character may be defined as an offset froma specified point (e.g., (0, 0)).

An “image update” primitive may comprise a reference to (e.g., a cacheID of) a region of the presenter's shared desktop, image data to beplaced in that region (e.g., an image object primitive), and anyinformation needed to unpack or decode the image data. At eachattendee's client, the image data are unpacked and the image isreproduced in the specified region.

In this embodiment, a “source Hit” action primitive is used when contentwithin the presenter's desktop is copied or moved from one position toanother. This often occurs when a window is moved or the contents of awindow are scrolled. A source blit in this embodiment identifies one ormore regions (e.g., by coordinates) and an offset representing how/wherethe content moved.

When a source blit is identified during the analysis of the presenter'sdesktop or received from an operating system or display driver, a regionmay be defined to fit the content that moved, in which case only oneregion must be identified. A reference to the content (e.g., a cache IDof a region) is transmitted, along with the offset by which it moved.The receiving clients locate the specified content, which is alreadydisplayed and move it appropriately.

A “tile” action primitive may be used to facilitate replication of arepeating pattern in the presenter's desktop. For example, one regionobject primitive may be defined to comprise a target region of thedesktop (e.g., a rectangular area) that has the same content orappearance (e.g., the same wallpaper or background). This target regionneed not be contiguous. Then, a tile (e.g., a cluster, a defined region)containing the same content is identified and used repeatedly (e.g.,“tiled”) over the targeted region.

In an embodiment of the invention, one or more object and/or actionprimitives are assembled to form a primitive block, one or moreprimitive blocks are assembled to form a media block, and one or moreblocks are transmitted as a single desktop packet from the presenter'sclient. A media or primitive block may be compressed (e.g., according tozlib, LZ, or some other algorithm).

FIG. 8 depicts a presenter's desktop sharing environment as it may beimplemented in one embodiment of the invention. In this embodiment, toshare the presenter's client desktop (or a portion of the desktop), theclient executes operating system 802, desktop sharing module 804 (e.g.,an application or utility) and codec (coder/decoder orcompressor/decompressor) 806. The presenter's coder/decoder will act asan encoder or compressor. Attendees' clients will be configuredsimilarly to the environment of FIG. 8, but their coder/decoders willact as decoders.

The portion of the presenter's desktop (the “shared” desktop) that isreproduced for collaboration attendees is referred to herein as the“bound” area or portion of the desktop. The codec is informed of thesize (e.g., dimensions) of the shared desktop and the position of thebound portion. The shared desktop and the bound area are dynamic duringa collaboration.

As shown in FIG. 8, the operating system (e.g., a display driver) maynotify desktop sharing module 804 when the desktop is altered. This maycommence a desktop sharing cycle as described above.

Notifications from the operating system may be made in the form ofevent(s) 812. An event may be fired when window is opened or moved inthe bound area, an icon is selected, a cursor moves or the appearance ofthe bound area changes for some other reason or in some other way. Inanother embodiment of the invention, module 804 may commence a cycle onits own initiative.

When a cycle is commenced, desktop sharing module 804 informs codec 806by passing it various information, 814, that will allow the codec toidentify any changes in the desktop (e.g., to identify any objects thatchanged, and the actions involved).

Thus, the desktop sharing module may pass a description (e.g., thecoordinates) of the bound portion, a description or definition of aregion in the bound portion that changed or that encompasses thechange(s), a new cursor position, a cursor image, a description andposition of a window that moved, a set of data defining a character orimage, etc.

In one embodiment of the invention, desktop sharing module 804 mayreceive source blits directly from a display driver, and pass them tothe codec. And, as described above, module 804 may perform checksums ondesktop clusters. The contents of any clusters that have changed may besent to the codec as raw data (e.g., the values of the individual pixelsin the cluster).

From the information provided by the desktop sharing module, codec 806identifies the individual object and action primitives to be passed toother clients to allow them to reproduce the change(s). For example,within a set of clusters received from or identified by module 804,codec 806 may identify a set of character objects. It will createcorresponding object primitives describing or defining the objects (ifnot already cached), and cache them. It will also create a suitable setof action primitives (e.g., character update primitives) to fill theobject primitives with the correct color.

Thus, from data received from desktop sharing module 804, codec 806creates higher-level object and action primitives. The codec assemblesand returns one or more primitive blocks or media blocks 818, containingthose higher level primitives, to the desktop sharing module. Codec 806may include the ability to compress primitives, generate media blocksand streams, etc.

The desktop sharing module assembles media blocks into desktop packets820 and forwards them to the operating system for transport. Theoperating system, or an applicable communication protocol stack,constructs network packets 822 to forward the desktop packets toward theattendees.

As described in a previous section, the desktop packet(s) may betransmitted to a real-time collaboration server, where they may berouted to a control unit responsible for controlling the desktop sharingmode. The desktop packet(s) may then be redirected or repackaged fordissemination to the attendees' clients. At the attendees' clients, themedia blocks are inflated to retrieve the primitive blocks.

Thus, codec 806 produces the necessary primitives to be sent toattendees' clients to allow them to reproduce the bounded region of thepresenter's shared desktop.

In one alternative embodiment of the invention, operating system 802 ordesktop sharing module 804 includes a display driver layer, or thepresenter's desktop environment includes a display driver layer betweenthe operating system and the desktop sharing module. In this embodiment,the display driver layer may perform various functions.

For example, this layer may optimize source blits by focusing on thebeginning position and ending position of the content that moved orscrolled. Intermediate positions need not be reported to other clients.A display driver layer may also perform the comparison of a cluster'spresent checksum with a previous checksum to identify clusters thatchanged.

And, the display driver layer may trim or shrink a region identified ashaving changed. Illustratively, when a set of clusters comprising one ormore changes to the presenter's desktop is identified, the displaydriver layer may redefine the region of actual changes. The resultingregion may not exactly align with cluster boundaries, but will besmaller in area.

In one embodiment of the invention, zlib compression is used to compressmedia blocks in a desktop packet. The compression is stream-based; acompressor (the presenter's codec) compresses media blocks, and aninflator (an attendee's codec) inflates them. Zlib compression is alsostateful. That is, the inflator maintains a state for the decompressionand, in order to inflate a mid-stream media block, the inflator musthave inflated all preceding media blocks in the stream.

Ordinarily, this would make it difficult to add a new attendee after adesktop sharing mode of collaboration was initiated, because the newattendee's inflator would not have the current state. And, it would beinefficient to send every new desktop sharing attendee all the precedingdata.

Therefore, in one implementation of this embodiment of the invention,the presenter's encoder inserts a media sync modifier between mediablocks. The media sync modifier may be inserted every X media blocks, orwith a particular periodicity, or at some other interval. When aninflator encounters a media sync modifier, it resets its state.Therefore, any set of media blocks delimited by a pair of media syncmodifiers can be decompressed independently of any other media blocks.In this implementation, new attendees to a desktop sharing mode may bebrought into the collaboration to coincide with a media sync modifier.

In an embodiment of the invention, a presenter's client may supportmultiple zlib streams of media blocks. For each stream, an attendee'sclient will maintain a separate inflator, each of which will maintain anindependent state. Stream-specific media sync modifiers will resetdifferent streams, and a full media sync modifier may be provided toreset all streams.

In one embodiment of the invention, media blocks are segregated intodifferent streams based on the type of object primitives or actionprimitives they contain. Other criteria may be used in otherembodiments.

Illustratively, except for media sync modifiers, every media block sentby a presenter comprises one or more compressed primitive blocks. Amedia block also has a header identifying its zlib stream or channel,and the amount of data in the media block. In one embodiment, seven zlibstreams of compressed data and one stream of uncompressed data aresupported.

The presenter's encoder multiplexes media blocks from the zlib streamsinto a single data stream transported to the attendees' clients asdesktop packets. A decoder at an attendee client analyzes the datastream, one media block at a time. A media sync modifier will cause oneor more inflators to be reset. Otherwise, based on the type of mediablock (e.g., the zlib stream), it is fed to the appropriate inflator.Then the individual primitive blocks are retrieved, parsed and executed.

Illustratively, a media sync modifier is created by a presenter'sencoder on a configurable schedule (e.g., every ten seconds). A mediasync modifier may comprise an uncompressed media block containing aheader but no primitive blocks. The header identifies one or morestreams or channels. An attendee's client will recognize the structureof the media sync modifier and reset the identified stream(s). Theencoder resets its stream(s) after generating the media sync modifier.Resetting a stream may have the effect of stopping the deflator orinflator for that stream and restarting it or invoking a new one.

In one embodiment of the invention, a presenter's desktop sharing modulecommunicates to a codec changes to the bound area of the shared desktopby describing to the codec the composition or appearance of one or moreclusters. In this embodiment, the desktop sharing module may describechanges to the codec in terms of these clusters. A cluster may bedefined to be any useful size and shape (e.g., sixteen pixels by sixteenpixels square). Irregular shapes are allowable, and a cluster need notcomprise a single contiguous portion of the shared desktop.

In different embodiments of the invention, different methods may be usedto find or identify or define a region of the presenter's desktop. Forexample, a region may be defined with a bitmap. Thus, a region within asixteen pixel by sixteen pixel square area could be defined by a bitmap256 bits in size (i.e., one bit for each pixel). Illustratively, a pixelis given one value (e.g., zero) if the pixel is not part of the regionbeing defined, and a different value (e.g., one), if it is part of theregion. Thus, a bitmap may be used to define or identify the pattern ofa character within a cluster or other area. As described above, anaction primitive may specify an action to take with regard to thatregion (e.g., fill it with a color).

Another method of representing a region may employ horizontal orvertical scanlines. As this method might be applied for horizontalscanlines, each scanline has a specified height (e.g., in pixels), whichmay vary from scanline to scanline, and may extend the entire width ofthe shared desktop or bound portion of the desktop, or less than theentire width. Successive scanlines are adjacent (i.e., from the top tothe bottom of the screen, or vice versa). Each scanline comprises one ormore scans, with each scan being limited to identifying one rectangle orother object in a region.

The shorter the scanline height, the higher the resolution of thescanline representation and the more data needed to define the region.Thus, scanline representation may be better suited for a shared desktopcomprising large rectangular regions. A bitmap representation may bemore suitable for more detailed regions.

In one embodiment of the invention, a region (e.g., the desktopbackground) can be described to attendees' clients by defining theregion (e.g., by coordinates, boundaries) in an object primitive andthen executing an action primitive to fill (or tile) the region with thespecified color (or pattern).

A region containing character object primitives may be defined as justthe shapes of the characters. That is, the region may comprise justvectors or pixels defining the characters. That region may be defined inany suitable manner (e.g., scanlines, bitmap) in an object primitive,and be followed by an action primitive indicating which color(s) to useto fill the character regions.

An image object may be defined by bitmap, scanline, or virtually anyother method. An image update primitive for reproducing the imageprimitive may include a value for each pixel in the image, wherein thevalue is the index in the palette of the color of the pixel.

Table 3 identifies primitive packets that may be used in an embodimentof the invention. As described above, the presenter's encoder assemblesprimitive packets into media blocks. Media blocks are transported fromthe presenter's client within desktop packets.

TABLE 3 Primitive Packet Description/Contents Shared_Screen Dimensionsof shared desktop screen Shared_Screen&Region Dimensions of sharedscreen and bound area of the shared screen Bound_Area Dimensions of thebound area Character One or more character object primitivesCursor_Image_BW Defines a monochromatic cursor object primitive CursorImage_Color Defines a color cursor object primitive Image One or moreimage object primitives Palette One or more palette object primitivesRegion One or more region object primitives Character_Update One or morecharacter update action primitives Image_Update One or more image updateaction primitives Source_Blit Defines a source blit action primitiveSet_Cursor Instructs client decoder to set mouse pointer to specifiedcached cursor Palette_Add Colors to be added to specified cached paletteSurface_Add Add item to surface cache Mouse_Position_Abs Current mouseposition, relative to point (0,0) of the shared desktop screenMouse_Position_Rel Current mouse position, relative to previous mouseposition Mouse_Position_Hide Instructs client decoder to hide thecurrent mouse pointer Drop_Cache_Item Instructs client decoder to dropthe specified item (or all items) from the specified cache (or allcaches) Viewport Definition of recommended viewport into bound area;generally at least as large as the bound area, and no larger than theshared desktop screen

In other embodiments of the invention, more or fewer primitive packetsmay be defined than are listed in Table 3.

Control Unit Operations

In present embodiments of the invention, a control unit may beconfigured to perform one or more specialized operations to facilitate acollaboration session.

In one embodiment, a control unit for controlling a desktop sharing,whiteboard, or other collaboration mode may maintain a “virtual client”for tracking the status of the collaboration (e.g., the appearance of apresenter's shared desktop, the contents of a shared whiteboard). Thevirtual client may, in particular, maintain a virtual in memory image orrepresentation of a presenter's or host's client desktop, including anycaches, buffers or other objects needed to reproduce a shared desktop,whiteboard, etc. The collaboration status maintained by a virtual clientmay be termed a “virtual screen.”

A virtual client may be maintained to help clients of attendees thatjoin an in-progress desktop sharing collaboration quickly synchronizewith the shared desktop. The virtual client thus acts like a regularclient of the collaboration mode, in that it will receive the same dataas the other clients, and apply the data to mimic the presenter'sdesktop.

In another embodiment, a control unit that has a slow client (e.g., aclient that accepts or requests data at a notably slower rate than otherclients in the same mode) may take some action to control the size ofits output queue (i.e., the queue of collaboration data to be sent toits clients). For example, depending on how far behind in thecollaboration the slow client is (e.g., how many packets or how muchdata are available that the client has not yet received), the outputqueue may be “collapsed” for this client. When a control unit's outputqueue is collapsed for a client, it is treated like a new client and issent a copy of the control unit's virtual screen (if one exists for thecollaboration mode) instead of trying to send it every separate set ofdata in the queue.

A Virtual Client

In one embodiment of the invention, a control unit may employ a virtualclient to maintain a virtual screen for a desktop sharing collaborationmode, a whiteboard mode, or any other mode in which it is necessary ordesirable to provide new attendees with collaboration data received bythe control unit before the new attendees joined.

For example, in a desktop sharing mode, and as described in a previoussection, changes to a presenter's desktop (or a shared portion of adesktop) may be sent as incremental updates. Clients may need to havereceived and processed previous updates in order to apply a new updateif the updates are packaged using a form of compression or a desktopsharing protocol that is stateful. When a virtual screen is maintainedby a virtual client within a desktop sharing collaboration mode, thevirtual client will act almost identically to a true client. Thus, itwill maintain caches for object and action primitives, apply anynecessary decompression, reset media streams in response to media syncmodifiers, etc.

In a whiteboard collaboration, new attendees may expect or want to seethe whiteboard as it appears when they join, not just all changes madeto the whiteboard after they join. In a chat mode, a new attendee maydesire a summary of, or an extract from, a discussion that occurredbefore he or she joined. In a web co-browsing mode, a new attendee maywish to view some or all of the previously browsed pages. Other modesmay also benefit from a virtual screen.

In a collaboration mode such as a whiteboard mode, a virtual client maynot attempt to maintain a mirror image of what actual attendees' clientsare displaying. Instead, it may just store copies of some or allcollaboration data packets that updated the whiteboard. The state of thecollaboration could be recreated at any time by applying those packets.

In one embodiment, a virtual client interfaces with a control unit queueusing a data consumer object similar to the data consumer objectsdescribed previously for actual clients. The process by which thevirtual client receives collaboration data is thus very similar to themanner in which actual clients receive the data. The virtual client,however, may be resident within the control unit, and therefore needs novirtual communication channel.

A new data packet from a client of the presenter or host is received bythe responsible control unit through the data provider objectcorresponding to that client. The control unit queues the data fordissemination to other collaboration clients. For each client, acorresponding data consumer object maintains a pointer or reference intothe control unit's queue to identify the most recent data it hasrequested on behalf of its client or the next data to be requested. Thevirtual client's data consumer does the same.

When a client (or the virtual client) is ready to receive new dataplaced in the control unit queue, the corresponding data consumeradvances its pointer accordingly, and the control unit instructs theconsumer to send the data to the client. When the virtual client for adesktop sharing mode receives a new data packet, it decompresses thepacket, if necessary, and renders the data to update its virtual screen.For a whiteboard collaboration, the virtual client may just save thedata (or the entire packet).

When a new client joins an ongoing collaboration mode that has anassociated virtual screen, the control unit requests a copy of thevirtual screen from the virtual client. A list of new clients may bemaintained to identify all those that join before the virtual screen isreceived and sent to the new clients.

In response to a request for a copy of the virtual screen, the virtualclient may act like a presenter of the collaboration mode does when itsends a first view of its shared desktop. That is, the full shareddesktop, whiteboard, web page or other content is assembled (e.g., andcompressed) for transmission, including any caches, scripts or otherobjects needed to reproduce the content. The process of assembling acopy of a virtual screen for a client joining a shared desktopcollaboration mode may be termed “packing.”

The virtual client signals the control unit when the virtual screen isready to be sent. The control unit then instructs the new clients' dataconsumer objects to send the virtual screen to the clients. The newclients' data consumer objects are updated to point to the control unitqueue entry corresponding to the most recent data included in thevirtual screen.

A virtual client may receive multiple requests for a copy of a virtualscreen while it is packing a copy. For example, after one new clientconnects to a control unit, others may also join before the first one issent a copy of the virtual screen. The virtual client will continue withthe copy it started making for the first new client; that copy may besent to all the new clients.

In an embodiment of the invention in which a virtual screen ismaintained for a desktop sharing collaboration mode, new clients mayonly be added to the collaboration at particular times. As described ina previous section, desktop sharing updates from a presenter may bepackaged using stateful compression (e.g., zlib) and multiple mediastreams. Media sync modifiers may be added to periodically reset some orall of the media streams.

In this embodiment, a new client may only be added when all streams arereset. Illustratively, the presenter of a shared desktop collaborationmode may be configured to reset all streams (with an appropriate mediasync modifier) on a predetermined schedule (e.g., every ten seconds,every five seconds).

When the virtual client that is maintaining a virtual screen for ashared desktop receives a request for a copy of the virtual screen, itmay not begin packing the virtual screen until it receives and processesa media sync modifier resetting all media streams or channels. In oneembodiment, a virtual client may automatically start or schedule thepacking of its virtual screen every time it encounters a media syncmodifier resetting all streams.

A virtual client may include a worker thread having three distinctstates: idle, rendering and packing. When idle, the worker thread willimmediately respond to a new data packet by rendering it, or respond toa request for a copy of the virtual screen by packing it. If a new datapacket is received while the worker thread is in the packing state, itmay finish making its copy of the virtual screen before rendering thenew data. As described above, if the worker thread receives anotherrequest for a copy of the virtual screen while it is packing one, it mayignore the duplicate request.

In another embodiment of the invention, a virtual client may employ athread provided by the organizer it which the virtual client resides.For example, in organizer 202 of FIG. 2, a virtual client operating incontrol unit 222 b may call upon a pool of work threads such as workthread(s) 214.

While in the rendering state, it may be illegal for the virtual clientto receive another new data packet. In particular, the virtual client'sdata consumer object may be configured to only consume a new data packetafter the virtual client has finished the previous set of data.

In different embodiments of the invention, a virtual screen may bemaintained synchronously or asynchronously. When a virtual screen ismaintained synchronously, all new data are passed to the virtual client,and the virtual screen is updated, before the data are released toactual clients. The virtual client may not need a client object (e.g., adata consumer) in this scenario. When a virtual screen is maintainedasynchronously, the virtual client receives access to new data atsubstantially the same time that other clients receive access. In amulti-processor environment, asynchronous maintenance may be moreeffective.

Queue Collapsing

In one embodiment of the invention, a control unit may collapse its dataqueue for a slow client, depending on the length of the queue, how muchcollaboration data the client has not yet received, and/or othercriteria. By collapsing the queue, the control unit is able to restrainthe size of the queue and prevent a slow client from affecting the rateat which other clients receive data. In addition, a slow client may beable to catch up to other clients faster than it could otherwise.

If a control unit did not collapse its queue, it could grow unbounded.This would consume memory resources and complicate queue management. Ifa limit were placed on the size of the queue, but the queue wasn'tcollapsed by the time it reached this size, then the rate at which otherclients could receive collaboration data from the control unit would belimited by the speed at which the slowest client retrieved data.

When a control unit collapses its queue, a client at the tail end of thequeue (i.e., at the entry representing the oldest queued data) isupdated to the head of the queue, or at least nearer to the head. Morespecifically, the pointer used by the slow client's data consumer isupdated, and any data that are no longer needed by any clients (e.g.,data at or near the tail end of the queue) may be discarded.

In one method of queue collapsing, a virtual screen is sent to a slowclient to update it to a position at or near the head of the queue. Bypacking and sending the virtual screen, the slow client does not need toreceive any of the data between its former position in the queue and theentry representing the most recent set of data included in the virtualscreen.

In this method, a control unit's queue size may be limited in size(e.g., approximately 1,000 entries) or the amount of data that can bequeued at one time (e.g., 3-4 MB). When this size is reached, new data(e.g., from a data provider) cannot be added until one or more oldentries are consumed by the client(s) that has/have not yet done so.However, there may be other times, when the queue is not yet at itsmaximum size, when it may be beneficial to collapse the queue.

In one embodiment of the invention, a “hard” queue collapse is effectedwhen a maximum size of the queue is reached. In contrast, a “soft” queuecollapse may occur when the queue is less than its maximum size, but itwould be more efficient to update a slow client by sending it thevirtual screen rather than sending every intervening set of data in thequeue.

When hard queue collapsing is active, the queue must be collapsed whenit reaches its maximum size, regardless of how many slow clients areworking at the tail end of the queue. Illustratively, when a packet isadded to the queue, its size is checked.

During a hard collapse, the slowest client (or clients if more than oneare equally slow) will be busy receiving a set of data when the queuereaches its maximum size. That client is moved to a list of clients forwhich the system is waiting for a set of data to be delivered. When theslow client finishes those data and requests more, it will be moved to alist of clients (e.g., new clients) awaiting a virtual screen (describedabove). When a copy of the virtual screen is ready, it will be sent toall clients in the list, including the slow client.

The slow client's pointer into the control unit queue is updated to themost recent data included in the virtual screen. Any entries between theslow client's old position in the queue and the present position of thenext slowest client can then be discarded.

In one implementation of this embodiment, the queue may not yet be atits maximum size, but there may be one or a limited number of clients ator close to the tail end. If a virtual screen is packed for a newclient, the slow client(s) may be added to the new client list and thequeue may be collapsed anyway.

When soft queue collapsing is active, the queue is examined whenever aclient (e.g., the slowest client) requests another set of data. Thequeue may be examined to determine how much data would have to be sentto the client to make it current. This is compared to the size of thevirtual screen, or an estimate thereof (e.g., the size of the virtualscreen when it was last packed). If the virtual screen is smaller thanthe incremental data that would have to be sent, then the queue may becollapsed for the slow client.

FIG. 9 illustrates an embodiment of the invention in which soft queuecollapsing may be performed. In FIG. 9, control unit queue 902 is thedata output queue for a control unit configured to control acollaboration mode that employs a virtual client to maintain a virtualscreen. Queue 902 has head 904, where new collaboration data are addedfor clients, tail 906 and maximum size 908. If the queue reaches itsmaximum size, hard packet collapsing may be automatically implemented.

In the illustrated embodiment of the invention, the size of a virtualscreen is noted whenever it is packed (e.g., for one or more newclients). An initial estimate may be approximately 200 KB. A dynamicthreshold, threshold 910 in FIG. 9, is defined (e.g., as a pointer) toidentify the portion of the queue, starting from the head, that issubstantially similar in size to the computed size of the virtualscreen. The threshold may be updated when the virtual screen is packedand its size calculated.

Thus, for any clients that have not yet received all data betweenthreshold 910 and tail 906, it may be more efficient to collapse thequeue and send them a virtual screen. To this end, when a client (orjust a slow client) requests a next set of data, the position of thatnext set of data in control unit queue 902 may be determined relative tothreshold 910.

For example, the slowest client that consumes data from queue 902 may becurrently receiving data from the queue entry at the position indicatedby reference 912. When this client requests more data, the system maydetermine that it would be more efficient to pack and send the virtualscreen

Because it may be relatively expensive to pack a virtual screen, in oneembodiment of the invention, a slow client may be updated by sending ita previously packed virtual screen instead of making a new copy. In thisembodiment, when a (slow) client requests data from the control unitdata queue, the system determines whether a packed virtual screen isavailable. Each packed virtual screen may be stored until another iscreated.

If a packed virtual screen is available, the system determines whetherthe virtual screen is smaller in size than the amount of data in thequeue between the client's current position and the position of the mostrecent data included in the packed virtual screen. If so, a copy of thatvirtual screen is sent to the client.

FIG. 10 demonstrates a method of performing hard and/or soft collapsingof a control unit queue according to one embodiment of the invention.The illustrated method begins with the queue being idle; it may compriseany number of entries containing or representing collaboration data.

In state 1002, an event occurs. In particular, either new data arereceived for the queue (e.g., from a client's data provider) or aclient's data consumer requests more data for its client. If new dataare received, the method advances to state 1020. Otherwise, the methodcontinues at state 1004.

In state 1004, the control unit determines whether the requesting clientis currently in the “danger zone.” In the illustrated method, the dangerzone comprises the portion of the control unit queue beyond thethreshold described above. In particular, the threshold marks the queuelocation at which the queued data (from the head to the threshold)matches or is similar to the size or estimated size of the controlunit's virtual screen. The danger zone comprises the remainder of thequeue. If the client's data consumer is currently pointing to the dangerzone, the method advances to state 1008.

Otherwise, in state 1006, the next set of data (e.g., collaboration datapacket) in the queue is sent to the requesting client. The control unitthen returns to an idle state.

In state 1008, the control unit determines whether a virtual screen iscurrently available (e.g., packed and ready for transmission). If so,the method advances to state 1012.

Otherwise, in state 1010, the control unit sets a flag or otherwisenotes that the virtual screen should be packed the next time a mediasync modifier is received for all media streams. Or, a specific requestmay be issued to pack the virtual screen at the next opportunity. Themethod then proceeds to state 1006.

In state 1012, the control unit determines whether it should send theavailable (or most recent) virtual screen. Illustratively, if a packedvirtual screen is smaller in size than the full panoply of data thatwould have to be sent to the client to bring it to the same state as thevirtual screen, then the virtual screen should be sent. If an availablevirtual screen should be sent, the illustrated method continues at state1014; otherwise, it proceeds to state 1010.

In state 1014, the virtual screen is sent to the client and the queue iscollapsed accordingly. In particular, the client's pointer is updated tocorrespond to the most recent data included in the virtual screen. And,entries at the tail end of the queue may be discarded if all otherclients have already received them. The control unit then returns to anidle state.

In state 1020, the control unit must determine whether it should apply ahard queue collapse. Therefore, the control unit examines whether thequeue size has reached a hard limit. If not, the illustrated methodadvances to state 1030.

Otherwise, the maximum queue size has been reached, and in state 1022,the queue is collapsed. The client(s) at the end of the queue are placedin a list, or otherwise scheduled, to receive the virtual screen, andtheir queue pointers will be updated according to the most recent datain the virtual screen.

In optional state 1024, if a flag or other marker had been set torequest a virtual screen at the next full media sync, the control unitmay determine whether any clients are currently in the danger zone(described above). If not, the flag may be cleared to indicate that noclients presently need the virtual screen. The illustrated method thenadvances to state 1030.

In state 1030, the new data are queued. In state 1032, the virtualclient retrieves or receives the new data (e.g., via a data consumerobject).

In state 1034, the control unit (e.g., a virtual client) determineswhether the new data includes a media sync modifier specifying that allmedia streams are to be reset. If it does, the method proceeds to state1036; otherwise, the method enters an idle state.

In state 1036 the control unit determines whether a particular flag ormarker is set to request the virtual screen be packed for transmission.If not, the method proceeds to an idle state.

Otherwise, in state 1038, the control unit (e.g., a virtual client)packs the virtual screen data and any other objects or information(e.g., caches) needed to recreate the current state of the virtualscreen. The size of the packed virtual screen may be stored or noted.

In state 1040, the threshold described above may be updated. Inparticular, this threshold marks the location in the queue at which thecombined size of all queued data between the head of the queue and thethreshold equals or is greater than the size of the virtual screen. Thecontrol unit then returns to an idle state.

To help determine whether a control unit's data queue should becollapsed, in one embodiment of the invention, each entry in the queueis augmented with a measure of the number of bytes (e.g., ofcollaboration data) that have been processed by the control unit so far.The measure may or may not include the data in the present entry.

Thus, the first entry in a control unit's queue, after it isinitialized, is marked with the value zero. The second entry is markedwith the amount of data in the first entry. The third entry is markedwith the amount of data indicated in the first entry plus the amount ofdata in the second entry, and so on. This prevents the control unit fromhaving to add the amount of data in each entry when it needs to needs todetermine whether it would be help efficiency to collapse the queue.

In one alternative method of queue collapsing, queued data are analyzedand redundant or obsolete data are removed. For example, in a whiteboardcollaboration mode, if one entry in the queue represents a clearing ofthe whiteboard by the presenter, data between that entry and a slowclient that has not yet reach that entry may be dropped.

A Real-Time Collaboration Client

In one embodiment of the invention, a client capable of participating ina real-time collaboration is modified for the collaboration withoutrequiring the client to be rebooted. And, if the client is rebooted, theclient will not lose the ability to participate in the collaboration.

In a client computing device, a display driver is executed to driveoperation of a video display. In an embodiment of the invention, a tablefor facilitating function calls into the display driver is patched tocall a different module. This module completes the call to the displaydriver, but also applies the call to update the state of the real-timecollaboration. Thus, video operations that change the appearance of thevideo display are intercepted for use in the collaboration.

FIG. 11 is a block diagram of a client configured according to oneembodiment of the invention. Client 1102 includes video display 1110,which is driven by display driver 1112.

Physical device object 1114 and logical device object 1116 correspond tothe video display. Each device object includes a table (i.e., tables1124, 1126) designed to facilitate the lookup and invocation of displaydriver functions. Table 1124 may be considered the “primary” table,because it is used during operation of the client. Table 1126 may beconsidered a “secondary” or backup table, which may be activated inplace of, or copied over, the primary table when needed (e.g., when thevideo mode changes). These tables may ordinarily store addresses, indisplay driver 1112, of video functions offered by the display driver.

Thus, before implementation of an embodiment of the invention, tables1124, 1126 may be configured to lookup or make calls to functionsprovided by display driver 1112, as indicated by the dashed line.

When an embodiment of the invention is implemented, however,collaboration module 1120 is installed, which assembles redirectiontable 1122. And, table 1126 of logical device object 1116 is patched topoint to or refer to redirection table 1122 instead of display driver1112.

Collaboration module 1120 then alters the video mode of video display1110, by changing the refresh rate, screen size or resolution, colordepth, or some other parameter. In response to this alteration, table1126 is copied over table 1124 and is thereafter used to facilitatevideo operations. As one skilled in the art will recognize, changing thevideo mode will cause the video display to be reset, which will resetthe video state information maintained by the display driver.

When a call to display driver 1112 is received at redirection table1122, the redirection table forwards the call to display driver 1112 sothat the call is handled by the display driver and the video display isupdated as necessary. However, the redirection table also passes thecall or its effects to collaboration module 1120. Thus, after the videomode is changed and the video state reset, collaboration module 1120 isable to keep track of the state and appearance of the video display.

Collaboration module applies intercepted video function calls to updateor supplement the client's real-time collaboration, as necessary. Thus,if the intercepted call was designed to draw an object on the videodisplay, display driver 1112 will still do so. In addition,collaboration module 1120 can forward the operation to othercollaborators (e.g., as part of a shared desktop or whiteboard mode ofcollaboration).

In an embodiment of the invention, when the video mode of video display1110 is altered, and physical device object 1114 is reinitialized,collaboration module 1120 may create surface objects corresponding toany or all of the surface objects created by the physical device objectfor video operations. However, as only video operations performed on theprimary surface object are written to the video display, collaborationmodule 1120 may only create a primary surface object.

Illustratively, display driver 1112, physical device object 1114,logical device object 1116, collaboration module 1120 and redirectiontable 1122 may all be installed in client system memory (e.g., thekernel). Further, redirection table 1122 may be installed as part of, orseparate from, collaboration module 1120.

In one embodiment of the invention, a collaboration client is alsomodified to enable a method of intercepting video operations to beimplemented after the client reboots. In this embodiment, loader module1130 is installed on the client, and is configured to be loaded beforethe client's normal display driver. As one skilled in the art willappreciate, the Windows operating system, among others, provides thisfunctionality.

When loader module 1130 is loaded during a system boot, it beginsmonitoring operating system calls to load system images. Eventually, itidentifies and intercepts a call to load the display driver. It maydetect a call to load the display driver by comparing the names ofimages the operating system attempts to load with the name of thedisplay driver stored in a registry or other configuration file (e.g.,the Windows registry). For example, in a Windows operating systemenvironment, loader module 1130 monitors LoadSystemImage calls to theWindows executive module.

When a call to load the display driver is intercepted, the displaydriver is allowed to load, but the loader module replaces or modifiesthe return value so that the operating system will make video calls toredirection table 1122 instead of the display driver. Also, the loadermodule loads collaboration module 1120. The operating system thenconfigures the physical and logical device objects in its normal manner,but based on the contents of redirection table 1122.

In one embodiment of the invention, loader module 1130 is a relativelysimple programming module. Thus, it may just load collaboration module1130, and the collaboration module may intercept the call to load thedisplay driver, and perform any other processing necessary to enableinterception of video calls.

Therefore, in this embodiment, the loader module rarely, if ever, willneed to be updated or replaced. Any code that may be expected to needupdating or replacement may reside in the collaboration module. If thecollaboration module is updated, no change of video mode is required.This is because the collaboration module retains its accumulated stateinformation in such a format that the information can be transferred tothe substitute module (i.e., the state information need not be collectedfrom the operating system).

Yet further, the collaboration module may comprise multiple sub-modules,so that code not expected to change may reside in one set ofsub-modules, while code that may need to be updated or replaced mayreside in another set of sub-modules.

FIG. 12 demonstrates a method of configuring a client for participatingin a real-time collaboration, according to one embodiment of theinvention. In this embodiment, if the client is rebooted, the client isreconfigured to maintain compatibility with the collaboration.

In state 1202, a collaboration module and loader module are installed onthe client. Illustratively, the collaboration module may be installedand executed when the client connects to a real-time collaborationserver, or other system, in order to join a collaboration. Thecollaboration module may install the loader module. The loader module isconfigured to be loaded into system memory, before a video displaydriver, if and when the client is rebooted.

In state 1204, a redirection table is created to receive directed orredirected calls to the video display driver. The redirection table maybe generated by the collaboration module. The redirection table isconfigured to forward an intercepted call to the display driver and tothe collaboration module.

In state 1206, a client table configured to facilitate calls to thedisplay driver is patched to direct or redirect such calls to theredirection table. Illustratively, this table is maintained by theclient operating system as part of a physical or logical device objectcorresponding to the client video display.

In state 1208, the collaboration module initiates a change to the videomode of the video display. Illustratively, the collaboration module mayfirst attempt to modify the display's refresh rate. If that isunsuccessful (e.g., the display is only configured to operate at onefrequency), the collaboration module will attempt to change a differentparameter (e.g., screen size or resolution, color depth). In differentembodiments of the invention, virtually any dynamically modifiable videomode parameter may be altered.

In state 1210, the collaboration module receives and uses interceptedvideo calls for a real-time collaboration. For example, if the clientdesktop is being shared with other collaborators, the collaborationmodule may capture changes to the desktop and disseminate them to thecollaborators, as described in a previous section. Thus, thecollaboration module may comprise a desktop sharing module and/or acoder/decoder. Intercepted calls are also passed to the display driverso they may be completed normally.

In state 1212, if the client is rebooted, the method proceeds to state1214. Otherwise, the collaboration module continues to intercept callsto the display driver.

In state 1214, when the client is rebooted, the loader module is loadedinto system memory before the display driver is loaded.

In state 1216, the loader module loads the collaboration module and theredirection table is generated (e.g., by the collaboration module).

In state 1218, the loader module intercepts a call to load the displaydriver. As described above, to intercept the correct call, the loadermodule may compare identities of images the operating system attempts toload with an identity of the display driver stored in an operatingsystem registry or configuration file.

In state 1220, the display driver is allowed to load, but a return valuefrom the call is modified so that the operating system will configure aclient table designed to facilitate calls to the display driver to callthe redirection table instead of the display driver.

In state 1222, calls to the display driver are intercepted as they werebefore the client was rebooted. Thus, the collaboration module is ableto update a real-time collaboration without noticeably affecting normaloperation of the client video display. The method then ends.

In one embodiment of the invention, a real-time collaboration client isconfigured to participate in, or host, a desktop sharing collaborationmode. One form of a desktop sharing collaboration mode was described ina previous section. In this embodiment, the client is configured with adesktop sharing engine.

FIG. 13 is a block diagram of a desktop sharing engine according to thisembodiment of the invention. In FIG. 13, desktop sharing engine 1302includes several modules: object controller 1310, update controller1312, update packer 1314, mouse controller 1316, highlight controller1318 and input emulator 1320. Any or all of these modules may becombined or further divided.

Desktop sharing engine 1302 may implement multiple session objects, suchas session object 1304. Each session object includes a separate set ofthe modules listed above, for a separate desktop sharing session. Forexample, different applications or portions of a desktop may be sharedin different sessions.

Object controller 1310 manages desktop objects that are shared as partof the collaboration mode and that are visible to collaborationparticipants (e.g., windows, applications, desktop areas). In additionto these shared objects, there may also be disabled objects, which arenot visible to other participants, even if located in a region or areaof the desktop that is being shared. Thus, the following types ofobjects may be either shared or disabled: a window, an application, arectangular area of the display screen, the entire screen. Objectcontroller 1310 offers an interface that provides methods for: adding orremoving an object to/from a list of shared objects, adding or removingan object to/from a list of disabled objects, and calculating a currentshared region.

Because the object controller is responsible for managing lists ofshared and disabled objects, it is able to identify or define a boundregion for a desktop sharing collaboration. It can also indicate whenthe shape of the bound region has changed.

Update controller 1312 identifies apart of the client display screen inwhich content has changed, and can provide content (e.g., pixels) fromthe changed area. The update controller provides an interface tocollaboration module 1332, which is described further below. Anotherembodiment of a collaboration module was described above in conjunctionwith FIG. 11. The interface provided by update controller 1312 includesany or all of the following methods: return the current shared regionand corresponding screen update data, and return an event object that issignaled when a change occurs on the display screen.

Update packer 1314 obtains data from update controller 1312 andcalculates regions of the display screen in which content changed. Inone implementation, the client display screen is divided into clustersand the update packer calculates a checksum on the content in eachregion. Update packer 1314 identifies regions in which changes actuallyoccurred by comparing consecutive checksums. When the shared desktopchanges, updated content is copied to an off-screen buffer; everycluster whose checksum changed is added to the defined “changed region”comprising all changes.

When a change is noticed in the bound region, the update packer is asked(e.g., by session object 1304) for the new changed region (i.e., theregion encompassing all changes) and a set of raw source blits. Tosatisfy this request, the update packer does the following:

-   -   1) asks the update controller for screen updates and sources        blits since the last set of changes; the update controller        provides raw update region data and raw source blits in        response;    -   2) if the update packer retains an update region from a previous        transaction, combines that region with the new update region        after raw source blits are applied; excludes the invalid region        from the update region (the invalid region includes area(s) of        the screen being repainted or awaiting repainting);    -   4) calculates the region actually changed (e.g., by comparing        cluster checksums, as described above); and    -   5) reports the changed region and raw source blits.

Mouse controller 1316 tracks the position and shape of the mouse cursor.Illustratively, the mouse controller may use a polling algorithm thatperiodically calls the GetCursorPos function to track cursor position.It may also periodically call the GetCursor function to track changes inthe cursor shape. A change in cursor shape may be detected by comparingresponses to consecutive GetCursor calls. The interface provided bymouse controller 1316 includes methods for: checking if the cursorposition has changed and returning the current position if it haschanged, and checking if the cursor shape has changed and returning thecurrent shape if it has.

Highlight controller 1318 provides visual feedback regarding a sharedarea of the desktop. In this embodiment of the invention, there arethree types of visual feedback. First, a shared area of the displayscreen that is not bound to any window is visualized with a two-pixelwide color border. The border may be implemented as a non-rectangulartopmost window. Second, shared windows are visualized with one-pixelwide color borders that may be implemented with a system-wide messagehook that modifies processing of the WM_NCPAINT message.

Third, a special button is added to each window title. This buttonallows a user to share or stop sharing that window with a single mouseclick. The button's appearance changes, depending on whether the windowis currently being shared. This functionality may be implemented using asystem-wide message hook that modifies processing of WM_NCPAINT andWM_NCACTIVATE messages.

An interface provided by highlight controller 1318 provides methods to:set the current highlighting mode (all three feedback methods can beturned on/off independently) and retrieve the current highlighting mode.

Input emulator 1320 emulates input events (e.g., cursor movements,content changes) received from remote entities (e.g., other clientsparticipating in the desktop sharing collaboration). Remote input eventsmay be restricted to the currently shared area of the desktop. Forexample, coordinates of remote mouse events may be compared to theshared region to determine whether the event should be implemented ordiscarded. Remote keyboard events are tied to particular windows. Agiven keyboard event is implemented only if its window lies entirelywithin the shared area of the desktop. Local input is usually givenpriority over remote input.

Illustratively, the desktop sharing engine or a session object may pollthe various modules for changes, and call coder/decoder 1330 to encodeinformation that has changed (e.g., the bound region, the changedregion, source blits cursor position, cursor shape).

A real-time collaboration client may initiate a desktop sharingcollaboration mode as follows. A session object is created and a sharedarea of the client desktop (e.g., one or more windows or applications)is associated with the session object. Various aspects of visualhighlighting of the shared area may be enabled or disabled (e.g.,highlighting of borders, window title buttons). The session object canthen start receiving screen updates and input events. Remote user inputevents may also be received. The session object is destroyed when thecollaboration mode ends.

In one embodiment of the invention, a real-time collaboration clientincludes a collaboration module to maintain, in system memory, an exactcopy of some or all of the content of client display screen.Illustratively, each display command or drawing request it receives isreplayed on the off-screen memory surface.

The collaboration module may operation in two phases. In an accumulationphase, display commands are processed and accumulated. In anormalization phase; accumulated data (e.g., updates caused by theaccumulated commands) are converted to update primitives and returned toupper layers (e.g., desktop sharing engine, update controller) forfurther processing.

The collaboration module may commence operating in the accumulationphase, during which drawing commands from applications are applied tothe display screen, and are intercepted by the collaboration module(e.g., via a redirection table such as redirection table 1122 of FIG.11). The intercepted commands are applied to an off-screen surface andcharacteristics of the commands may be saved (e.g., as a source blit orpart of the dirty region). When an upper layer requests the accumulateddata, the algorithm switches to the normalization phase and provides thecorresponding primitives to the upper layer in a suitable form. Thecollaboration module then returns to the accumulation phase.

More specifically, during the accumulation phase, the region of thedisplay screen affected by a drawing command (e.g., the “affectedregion”) is identified and merged with the current “dirty region.” Thedirty region is a region containing all points affected by drawingcommands (other than raw source blits) received since the beginning ofthe current accumulation phase. In the following normalization phase,bitmap copy primitives for reproducing pixels in the dirty region aresent to upper layers for distribution to remote collaborators. Theprimitives may be generated during the accumulation and/or normalizationphases. Raw source blits are treated differently—they are stored in anarray of raw source blits. The dirty region (or primitives forreproducing the dirty region) and the array of raw source blitsconstitute input to the normalization phase.

However, before being stored in the array, raw source blits receiveadditional processing. First, “dirty region propagation” is performed.In dirty region propagation, the destination position of a raw sourceblit is subtracted from the current dirty region because that part ofthe display will be overwritten by the source blit. Then theintersection of the source position of the raw source blit with thecurrent dirty region is found and propagated to the destination of theblit to determine where those “dirty” pixels will appear when the sourceblit is executed. The intersection can then be excluded from the sourcelocation of the blit.

Second, “source propagation” is performed to find the origin of anypixels involved in a sequence of multiple source blits. The sourceposition of the current raw source blit is intersected with thedestination position of the next oldest raw source blit in the array ofblits. If the intersection is non-empty, it is separated from theoriginal blit, propagated to the source of the preceding blit, andstored in the raw source blits array. The remainder is then intersectedwith the destination of the next oldest source blit in the array, and soon. This process continues until the source region of the original blitis empty or every raw source blit in the array is visited. In the latterevent, the remainder is appended to the source blits array.

As one skilled in the art will appreciate, propagation of a set ofpixels can be performed as follows. A region R of pixels may berepresented as a set of points (x, y). A source blit may be representedas a tuple in the form <D, dx, dy>, wherein D is a destination region(i.e., a set of pixels) of the blit, dx is a horizontal offset from thesource region and dy is a vertical offset. Thus, when a source blit isexecuted to change a display screen, every pixel (x, y) in a destinationregion D of a source blit is copied from a source region as follows:screen[x, y]:=screen x−dx, y−dy]. The source region need not bespecifically identified because it can be easily calculated using thedestination region and offsets.

To propagate a region R to the destination of a source blit allows oneto determine where the region would appear if it were moved with thepixels of the source blit. The original region R may be offset to regionR′ as follows:R′={(x+dx,y+dy)|(x,y)εR}

To propagate a region R to the source of a source blit allows one todetermine where the source pixels lie:R′={(x−dx,y−dy)|(x,y)εR}

Thus, in one embodiment of the invention, an accumulation phase mayproceed as indicated in the following pseudo-code:

Dirty := Ø; RawBlits := Ø; for each primitive drawing command P, do:  ifP is not a raw source blit then   R := region affected by P;   Dirty :=Dirty ∪ R;  else   S := source(P);   D := destination(P);      {Dirtyregion propagation}   X := Dirty ∩ S;   Dirty := (Dirty \ D) ∪ X;   S :=S \ X;      {Source propagation}   for i := top(RawBlits) to 0 do    X:= destination(RawBlits[i]) ∩ S;    if X ≠ Ø then     offset X to thesource of RawBlits[i];     append X to end of RawBlits array;    end if   S := S \ X;    if S = Ø then end for;   end;   if S ≠ Ø then append Sto end of RawBlits array;  end if; end for;

In the normalization phase, an output stream of update primitives isgenerated to be sent to remote clients participating in the desktopsharing collaboration. Raw source blits are processed to enableunambiguous replay on remote clients. Then pixels belonging to the dirtyregion are captured and converted into bitmap copy primitives.Illustratively, the source blit primitives are streamed first, followedby the bitmap copy primitives.

As indicated above, the destination location of a source blit may beaffected by a subsequent drawing command. In other words, the blitdestination may intersect the dirty region or the destination of anothersource blit. This intersection can be omitted from the source blitprimitive because it will be filled by bitmap copy primitives or asubsequent blit.

A collision detection scheme may be applied to detect collisions betweensource blits. When a source blit is fetched from the raw source blitsarray to be added to the output stream, source regions of remainingblits in the array are tested for intersection with the destination ofthe current blit. If an intersection is non-empty, it is subtracted fromthe corresponding bit and added to the dirty region.

Thus, a normalization phase may operate in a manner similar to thefollowing pseudo-code:

Primitives :=Ø;

for i :=top(RawBlits) to 0, do:

-   -   D :=destination(RawBlits[i]);    -   D :=D \ Dirty;    -   for j :=i−1 to 0 do        -   destination(RawBlits[j]):=destination(RawBlits[j]) \ D;        -   X :=source(RawBlits[j]) ∩ D;        -   X :=offset X to the destination of RawBlits[j];        -   Dirty :=Dirty ∪ X;    -   end;    -   if D≠Ø then append RawBlits[i] to end of Primitives;

end;

The foregoing descriptions of embodiments of the invention have beenpresented for purposes of illustration and description only. They arenot intended to be exhaustive or to limit the invention to the formsdisclosed. Accordingly, the above disclosure is not intended to limitthe invention; the scope of the invention is defined by the appendedclaims.

1. In a real-time collaboration among multiple collaboration clients, amethod of tracking the status of the collaboration, the methodcomprising: receiving, at a real-time collaboration server from a firstcollaboration client, data submitted as part of a real-timecollaboration involving multiple collaboration clients; queuing the dataat a control unit of the collaboration server for dissemination to theother collaboration clients; operating a virtual client at the controlunit of the collaboration server, wherein the virtual client acts as acollaboration client and receives the same queued data from the controlunit as the other collaboration clients; maintaining, by the virtualclient, a collaboration status in a virtual screen that comprises arepresentation of the real-time collaboration; updating therepresentation within the virtual screen to include the received dataand to track the collaboration status of the real-time collaboration;establishing a connection with a new collaboration client; generating acopy of said representation of the real-time collaboration which tracksthe updated status of the real-time collaboration; and transmitting saidcopy to the new collaboration client to facilitate synchronization ofthe new collaboration client with the real-time collaboration.
 2. Themethod of claim 1, wherein said receiving further comprises: passingsaid data to a virtual client executing within the real-timecollaboration server; wherein said virtual client is configured tomaintain said representation.
 3. The method of claim 1, wherein saidrepresentation comprises the virtual screen configured to track theappearance of a desktop of the first collaboration client.
 4. The methodof claim 1, wherein said updating comprises: adding the data to acollection of data, wherein a current state of the real-timecollaboration can be recreated by applying said collection of data. 5.The method of claim 1, wherein said updating comprises: rendering thedata within said representation to reflect a current visual appearanceof the real-time collaboration.
 6. The method of claim 1, wherein saidgenerating comprises: packing said representation of the status of thereal-time collaboration into a form that can be applied by the newcollaboration client to reproduce said status.
 7. The method of claim 6,wherein said status comprises graphical content of a shared clientdesktop.
 8. The method of claim 1, further comprising: queuing the dataas an entry in a queue of collaboration data released to collaborationclients; and for each collaboration client, maintaining a reference tothe queue entry comprising data currently being processed by thecollaboration client.
 9. The method of claim 8, further comprising:transmitting said copy to a first collaboration client; and updatingsaid reference for said first collaboration client to refer to thenewest data included in said copy.
 10. A non-transitory computerreadable storage medium storing instructions that, when executed by acomputer, cause the computer to perform a method of tracking the statusof a real-time collaboration involving multiple collaboration clients,the method comprising: receiving, at a real-time collaboration serverfrom a first collaboration client, data submitted as part of thecollaboration; queuing said data at the control unit of thecollaboration server for dissemination to the other collaborationclients; operating a virtual client at the control unit of thecollaboration server, wherein the virtual client acts as a collaborationclient and receives the same queued data from the control unit as theother collaboration clients; maintaining, by the virtual client, acollaboration status in a virtual screen that comprises a representationof the real-time collaboration; updating the representation within thevirtual screen to include the received data and to track thecollaboration status of the real-time collaboration; establishing aconnection with a new collaboration client; generating a copy of saidrepresentation of the real-time collaboration which tracks the updatedstatus of the real-time collaboration; and transmitting said copy to thenew collaboration client to facilitate synchronization of the newcollaboration client with the real-time collaboration.
 11. A method ofmonitoring the status of a real-time collaboration in which apresenter's desktop is shared with multiple clients, comprising:operating a virtual client at a control unit of a real-timecollaboration server configured to manage a real-time collaboration inwhich a presenter's desktop is shared with other collaboration clients,wherein the virtual client acts as a collaboration client and receivesthe same queued data from the control unit as the other collaborationclients; receiving a first set of data from the presenter at the virtualclient, wherein said a first set of data comprises one or more mediastreams; queuing the first set of data at the control unit of thecollaboration server; maintaining, by the virtual client, acollaboration status in a virtual screen within which the first set ofdata is rendered to mimic the presenter's desktop; receiving a mediasync modifier configured to trigger a reset of said media streams; andin response to said media sync modifier, packing said virtual screen fortransmission to the other collaboration clients.
 12. The method of claim11, wherein said packing comprises: generating a communicationconfigured to facilitate reproduction of the virtual screen on a client.13. The method of claim 11, further comprising: forwarding said packedvirtual screen to a new client joining the real-time collaboration. 14.The method of claim 11, further comprising: forwarding said packedvirtual screen to one or more of the multiple clients.
 15. The method ofclaim 11, wherein said virtual client is configured to maintain saidvirtual screen as a copy of content displayed on the presenter'sdesktop.
 16. The method of claim 11, wherein the first set of datacomprises graphical data representing a change in the appearance of thepresenter's desktop.
 17. The method of claim 16, further comprisingrendering said change in said virtual screen.
 18. The method of claim17, wherein said rendering comprises updating a cache of graphicalobjects cached by the presenter.
 19. The method of claim 11, wherein thefirst set of data comprises one or more of audio data and video data.20. The method of claim 11, further comprising: maintaining a queue insaid real-time collaboration server, wherein each entry in said queuecomprises collaboration data distributed to the multiple clients;maintaining, for each of the multiple clients, a reference identifyingthe queue entry comprising data currently being processed by the client;and adding the first set of data to a first entry in said queue.
 21. Themethod of claim 20, further comprising, if the queue reaches a maximumsize: identifying a slow client referencing the oldest entry in thequeue; sending said packed virtual screen to the slow client; andupdating the slow client's reference to identify the first entry in thequeue.
 22. The method of claim 20, further comprising: receiving arequest from a slow client for the data in a next entry in the queue;comparing a size of said packed virtual screen to a total size of alldata in the queue from said next entry to said first entry; and if saidtotal size exceeds said size of said packed virtual screen: sending saidpacked virtual screen to the slow client; and updating the slow client'sreference to identify the first entry.
 23. A non-transitory computerreadable storage medium storing instructions that, when executed by acomputer, cause the computer to perform a method, the method comprising:operating a virtual client at a control unit of a real-timecollaboration server configured to manage the real-time collaboration inwhich a presenter's desktop is shared with other collaboration clients,wherein the virtual client acts as a collaboration client and receivesthe same queued data from the control unit as the other collaborationclients; receiving a first set of data from the presenter at the virtualclient, wherein said a first set of data comprises one or more mediastreams; queuing the first set of data at the control unit of thecollaboration server; maintaining, by the virtual client, acollaboration status in a virtual screen within which the first set ofdata is rendered to mimic the presenter's desktop; receiving a mediasync modifier configured to trigger a reset of said media streams; andin response to said media sync modifier, packing said virtual screen fortransmission to the other collaboration clients.